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BUILD  provides  a  static  framework  for  modeling  systems  and  handling  construction 
requests  that  makes  use  of  programming  environment  specific  definitions.  By 
altering  the  set  of  definitions,  BUILD  can  be  extended  to  work  with  new 
programming  environments  and  to  perform  new  tasks. 
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Abstract 

y  BUILD  is  a  tool  for  keeping  modular  systems  in  a  consistent  state  by  managing  the  construction  tasks  (e.g. 
compilation,  linking  etc.)  associated  with  such  systems.  It  employs  a  user  supplied  system  model  and  a 
procedural  description  of  a  task  to  be  performed  in  order  to  perform  the  task.  This  differs  from  existing  tools 
which  do  not  explicitly  separate  knowledge  about  systems  from  knowledge  about  how  systems  are 
manipulated. 
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1.  Introduction 


Many  programming  languages  encourage  the  development  of  modular  systems  by  allowing  the 
independent  compilation  of  modules  (ADA  [Ada  83],  C  [Kemighan  and  Ritchie  78],  CI.U[I.iskov  81], 
Common-1. isp  [Steele  84).  Mesa  [Mitchell  79]).  This  feature  can  be  exploited  to  minimize  the  amount  of 
compilation  that  needs  to  be  done  when  some  part  of  a  system  is  changed.  However,  as  systems  become 
larger  it  becomes  difficult  to  know  exactly  which  modules  need  to  be  recompiled  when  one  changes.  It  is 
important  that  the  correct  modules  be  recompiled  and  relinked  --  a  bug  caused  by  ignoring  a  module  that 
should  be  rebuilt  can  be  very  difficult  to  find.  This  problem  is  called  the  consistent  construction  problem. 

This  report  describes  BUILD.  a  tool  that  reconstructs  system  modules  in  order  to  ensure  that  they  are  kept 
in  a  consistent  state.  BUILD  docs  not  modify  source  modules  and  will  not  rid  systems  of  problems  that  require 
source  code  revision.  However,  BUli.l)  can  handle  the  many  instances  where  some  portion  of  a  system  needs 
to  be  recompiled,  relinked,  or  somehow  reprocessed  in  order  to  eliminate  inconsistency. 

There  are  many  tools  that  manipulate  systems  by  reconstructing  inconsistent  parts.  Chapter  2  presents 
maki:  [Feldman  79j  and  DITSYSTIM  [Weinreb  and  Moon  81],  two  representative  tools,  and  discusses  some  of 
their  weaknesses.  The  fundamental  problem  with  makh,  DHl'SYSITM.  and  all  similar  construction  directive 
based  tools  is  that  they  operate  on  systems  by  using  user  supplied  lists  of  construction  directives.  These  lists 
arc  difficult  to  understand.  BUILD  provides  the  same  functionality  as  existing  tools  but  docs  so  without 
requiring  users  to  list  construction  steps. 

HUU.D  derives  the  construction  steps  needed  to  produce  a  module  from  user  supplied  system  models. 
These  models  specif  y  how  modules  reference  each  other  instead  of  how  they  arc  constructed,  build  uses  the 
reference  information  to  determine  how  modules  depend  on  each  other  and  how  a  change  to  one  module  will 
effect  another.  For  instance,  if  a  system  model  specifies  that  module /  refers  to  macros  defined  in  module 7  then 
build  can  infer  that  a  change  to  module2  implies  that  module ;  should  be  recompiled.  Chapter  3  discusses 
system  models  and  chapters  4,  and  5  explain  how  BUILD  uses  system  models  to  perform  construction. 

The  major  strength  of  build’s  reference  based  modeling  system  over  a  construction  directive  based  system 
is  that  it  provides  a  higher  level  language  for  describing  system  structure.  Because  it  eliminates  low  level 
construction  detail  and  allows  explicit  declaration  of  high  level  system  relationships,  a  reference  based  mode) 
is  easier  to  understand  and  provides  more  information  than  its  construction  directive  based  counterpart. 

build  separates  knowledge  about  systems  from  knowledge  about  how  systems  are  manipulated.  The  term 
task  is  used  to  refer  to  a  construction  process  such  as  compilation  or  linking  that  build  may  be  called  upon  to 
perform,  build  uses  task  descriptions  to  specify  how  to  perform  construction  tasks  and  how  the  various  kinds 
of  references  that  appear  in  system  models  may  effect  the  construction  required  to  perform  the  task.  Using 
the  example  from  the  previous  paragraph,  build’s  task  description  for  compilation  allows  it  to  realize  that 
while  a  change  to  module 2  implies  that  module /  should  be  recompiled,  a  change  to  module {  does  not  imply 
that  module 2  should  be  recompiled. 


TINYCOMP 

i  IN vcovii*  is  an  example  of  a  modular  system,  it  will  be  used  throughout  this  report  to  present  different 
aspects  of  system  construction  tools  (this  example  was  adapted  from  one  used  hy  Feldman  [Feldman  79]). 
TINYCOMP  has  two  major  modules,  a  parser  and  a  code  generator.  The  parser  is  built  by  yacc,  a  parser 
generating  tool  [Johnson  78a|.  The  code  generator  is  implemented  in  CJKcrnighan  and  Ritchie  78],  Ihc 
parser  and  code  generator  use  a  common  set  of  definitions  for  shared  data  structures.  These  definitions  are 
combined  with  the  source  programs  during  compilation.  The  compiled  programs  arc  linked  with  a  library 
that  is  also  subject  to  change.  Figure  1-1  depicts  tinycomp’s  inter-module  reference  pattern  and  figure 
1-2  depicts  TIN YCOMP's  construction  process. 


Figure  M:  TINYCOMP  Inter-Module  Reference  Graph 


Figure  1-2:  Construction  Graph  For  llNYCOMP 


Reference  Rased  System  Models 

Compare  figure  1-3  which  contains  the  maki  directives  for  tinycomi’,  and  figure  1-4  which  contains  the 
m  ill)  system  model  for  tinycomi*.  While  the  Maki  directives  encode  TTNYCOMP's  construction  graph, 
null  l)'s  system  model  encodes  i  inycomp's  reference  graph. 

A  reference  model  can  be  used  in  place  of  a  construction  directive  list  because  all  of  the  information  about 
construction  present  in  such  a  list  can  be  derived  from  a  reference  model.  Consider  the  third  maki-  directive 
for  tinycomi*: 

C00EGEN.0:  CODEGEN. C  DE F IN  IT  IONS . C 

CC  -C  CODEGEN. C  #  -C  COMPILES 

Ibis  expresses  that  CODEGEN.  0  is  produced  by  compiling  CODEGEN.  C.  and  that  if  either  CODEGEN.  C  or 
DEFINITIONS. C  changes,  then  CODEGEN. C  needs  to  be  recompiled.  This  construction  dependency  exists 
because  CODEGEN  .  C  is  combined  with  DEFINITIONS .  C  when  it  is  compiled  to  produce  CODEGEN  .0. 

In  contrast,  the  reference  based  model  specifics  that  CODE-GENERATOR  includes  DEFS: 

(: INCLUDES  CODE-GENERATOR  DEFS) 

BUILD'S  description  for  compilation  contains  the  knowledge  that  the  :  INCLUDES  reference  implies  a 
compilation  construction  dependency  between  including  and  included  files. 

PARSER. C:  PARSER . GRAMMAR 

YACC  PARSER. GRAMMAR  #YACC  MAKES  Y.TAB.C 

MV  Y.TAB.C  PARSER. C  ^RENAME  Y.TAB.C 

PARSER. 0:  PARSER. C  DEFINITIONS. C 

CC  -C  PARSER. C  *  -C  COMPILES 

CODEGEN. 0:  CODEGEN. C  DEFINITIONS. C 

CC  -C  CODEGEN. C  #  -C  COMPILES 

TINYCOMP:  CODEGEN. 0  PARSER. 0  LIBRARY. 0 

CC  CODEGEN. 0  PARSER. 0  LIBRARY. 0  -0  TINYCOMP  #  -0  LINKS 

Figure  1-3:  Makefile  For  tinycomp 


(DEFMODEL  TINYCOMP 

( : MODULE  DEFS  :C-S0URCE  "DEFINITIONS") 

( : MODULE  PARSER  : YACC-GRAMMAR  "PARSER") 

( : MODULE  CODE-GENERATOR  :C-S0URCE  "CODEGEN") 
( : MODULE  LIBRARY  :C-0BJECT  "LIBRARY") 


INCLUDES  PARSER  DEFS) 

INCLUDES  CODE-GENERATOR  DEFS) 

CALLS  PARSER  LIBRARY) 

CALLS  PARSER  CODE-GENERATOR) 

CALLS  CODE-GENERATOR  LIBRARY)) 

Figure  1-4:  BUILD  Model  For  t  inycomp 
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Task  Descriptions 

l  pon  receipt  of  a  request  to  perform  a  t.isk.  mil  I)  derives  a  task  graph  which  models  the  construction 
steps  and  dependencies  necessary  to  perform  the  task.  (Chapter  4  presents  lit  ll  n  task  models  and  chapter 
5  explains  how  task  models  arc  derived  from  system  models.)  Once  the  task  model  has  been  derived,  mill) 
analv/es  it  in  order  to  determine  which  components  have  changed  and  what  steps  are  needed  in  order  to 
satisfy  the  task  request. 

m  ll  l)  provides  a  static  framework  for  modeling  systems  and  handling  construction  requests  that  makes 
use  of  programming  environment  specific  definitions.  New  tasks  can  be  added  to  m 'll  l)s  repertoire  by 
altering  the  set  of  definitions. 

for  example,  figure  1-5  contains  the  forms  needed  to  define  a  task  called  :  LIST-SOURCE-CODE  which 
produces  formatted  listings  of  the  source  modules  of  a  l  isp  system.  (This  example  will  be  explained  in  detail 
in  chapter  5.)  The  first  form  allows  mil  l>  to  represent  the  processing  needed  to  list  a  single  Lisp  source  file. 
I  he  second  form  tells  in  II .n  what  to  do  when  a  :  LIST -SOURCE-CODE  request  is  received.  The  last  two 
forms  tell  UUII)  about  the  implications  of  the  references  :CALLS  and  :MACRO-CALLS  upon  the 
:  L I  ST  -  SOURCE  -  CODE  task. 

Since  task  definitions  arc  separate  from  system  models,  new  tasks  can  be  performed  on  existing  models 
without  additional  effort.  Tor  instance,  once  :  LIST-SOURCE-CODE  has  been  defined,  mil  l)  will  be  able  to 
handle  requests  to  format  the  source  code  for  existing  systems  without  changing  any  system  models. 
Construction  directive  based  tools  cannot  be  extended  in  a  similar  manner. 


(DEFINE -PROCESS- TYPE  : LIST-LISP-SOURCE 
((SOURCE  : LISP-SOURCE  : SINGLE  ) ) 

((LISTING  : PRESS  :SINGLE  SOURCE)) 

OUTPUT-STREAM 

(FORMAT  OUTPUT-STREAM  "~%LIST  ~A" 

(PATHNAME-MINUS-VERSION  SOURCE)) 

(FORMAT  OUTPUT-STREAM  "-ZLISTING  -A"  SOURCE) 

(LIST-LISP-FILE  SOURCE  LISTING)) 

(DEFINE-REQUEST-HANDLER  (: LIST-SOURCE-CODE  :LISP-SOURCE  :PRE) 

(SOURCE-NODE) 

(ACCESS*  SOURCE-NODE  ((SOURCE  : L IST-L ISP-SOURCE )  LISTING))) 

(DEFINE-REFERENCE-HANDLER  ((: MACRO-CALLS  :LISP-SOURCE  : LISP-SOURCE) 

( :LIST-SOURCE-CODE  :  LEFT ) ) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  : L 1ST -SOURCE -CODE  CALLED-NODE)) 

(DEFINE-REFERENCE-HANDLER  ((.CALLS  :LISP-SOURCE  :LISP-SOURCE) 

( :LIST-SOURCE-CODE  : LEFT ) ) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  : LIST-SOURCE-CODE  CALLED-NODE)) 


Figure  1-5:  Definition  For  : LIST-SOURCE-CODE 
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2.  System  Construction  Fools 

This  chapter  focuses  on  two  tools  that  were  designed  to  aid  in  the  management  of  the  consistent 
construction  problem.  Before  they  arc  presented  some  terminology  that  will  be  used  throughout  this  report  is 
introduced. 

Different  programming  environments  arc  geared  to  operate  upon  different  kinds  of  objects,  lor  instance, 
some  environments  arc  designed  to  operate  on  files,  and  others  on  functions.  The  term  grain  will  be  used  to 
refer  to  the  objects  manipulated  in  a  programming  environment  --  regardless  of  their  nature. 

The  terminology  introduced  in  this  paragraph  will  be  used  to  refer  to  the  kinds  of  grains  that  are 
manipulated  during  the  construction  process.  Source  grains  are  the  components  that  arc  produced  by  people 
and  not  programs  (c.g.,  programming  language  source  code).  Source  grains  arc  manipulated  by  programs  to 
produce  derived  grains  (c.g.,  object  code).  Grains  that  arc  the  final  products  of  the  construction  process  are 
called  goal  grains  (c.g.,  executable  images  of  programs).  While  goal  grains  arc  usually  derived  grains,  they  can 
also  be  source  grains.  Derived  grains  that  arc  not  goal  grains  arc  called  intermediate  grains  (c.g.,  object  code 
that  requires  linking  in  order  to  form  executable  images). 


2.1  MAKE 

make  [Feldman  79],  available  as  part  of  UNIX1,  is  a  simple  tool  for  managing  systems  that  has  received 
widespread  use.  maki-  is  driven  by  sets  of  construction  directives  dial  form  "recipes"  for  constructing 
systems.  These  directives  arc  stored  in  a  text  file  called  a  MakeFile  and  have  the  form: 

TARGET-GRAIN  :  INGREDIENT-GRAIN-1  INGREDIENT-GRAIN-2  ... 

COMMAND -1 
COMMAND-2 


Each  entry  declares  that  TARGET-GRAIN  depends  on  each  of  the  grains  to  the  right  of  the  colon.  The 
command  sequence  below  the  construction  dependency  declaration  line  is  executed  in  order  to  construct 
TARGET-GRAIN.  There  arc  no  constraints  placed  on  the  commands  which  can  appear  in  the  command 
sequence.  Furthermore,  there  arc  no  ordering  rules  for  MakeFile  entries. 

make  has  a  simple  macro  substitution  facility.  A  macro  is  defined  in  the  following  manner: 

MACRO- NAME= MACRO- EXPANSION 

Any  instance  of  MACRO-NAME  enclosed  within  parentheses  and  preceded  by  a  dollar  sign  (i.e., 
S(MACRO-NAME))  is  replaced  by  the  text  MACRO-EXPANSION  when  the  MakeFile  that  includes  the  macro 
definition  is  processed.  The  definition  for  a  macro  must  precede  all  of  its  uses. 


A  Small  Example  ••  T1NYCOMP 

Figure  2-1  depicts  the  construction  process  for  TINY  COME  and  figure  2-2  contains  a  corresponding 
MakeFile.  Given  the  MakeFile.  MAKI  will  perform  the  appropriate  construction  when  TINYCOMP 
components  change.  For  instance,  a  change  to  PARSER. GRAMMAR  will  cause  a  new  parser  to  be  derived, 
compiled,  and  linked.  A  change  to  CODEGEN. C  will  cause  CODEGEN. C  to  be  compiled  and  linked.  A 
change  to  DEFINITIONS. C  will  cause  PARSER. C  and  CODEGEN. C  to  be  compiled  and  linked.  Finally,  a 
change  to  LIBRARY  .0  will  cause  linking  but  no  compiling. 


UNIX  is  a  trademark  of  Hell  I  aboratorics 


Figure  2*1:  Construction  Graph  For  tinycomp 

PARSER. C:  PARSER. GRAMMAR 

YACC  PARSER. GRAMMAR  #YACC  MAKES  Y.TAB.C 

MV  Y.TAB.C  PARSER. C  ^RENAME  Y.TAB.C 

PARSER. 0:  PARSER. C  DEFINITIONS. C 

CC  -C  PARSER. C  #  -C  COMPILES 

CODEGEN. 0:  CODEGEN. C  DEFINITIONS. C 

CC  -C  CODEGEN. C  #  -C  COMPILES 

TINYCOMP:  CODEGEN. 0  PARSER. 0  LIBRARY. 0 

CC  CODEGEN. 0  PARSER. 0  LIBRARY. 0  -0  TINYCOMP  #  -0  LINKS 

Figure  2*2:  Makefile  For  TINYCOMP 
The  Makefile  entries  arc  interpreted  in  the  following  manner: 

PARSER. C:  PARSER. GRAMMAR  ... 

PARSE  R.C  depends  on  PARSER.  GRAMMAR.  It  is  created  by  running  YACC  on  PARSER.  GRAMMAR. 

PARSER. 0:  PARSER. C  DEFINITIONS. C  ... 

PARSE  R.O  depends  on  PARSER  .C  and  DEFINITIONS  .C.  It  is  created  by  recompiling  PARSER.  C. 

CODEGEN. 0:  CODEGEN. C  DEFINITIONS. C  ... 

CODE  GEN  .0  depends  on  CODEGEN  .C  and  DEFINITIONS.C.  It  is  created  by  recompiling  CODEGEN  .C. 

TINYCOMP:  CODEGEN. 0  PARSER. 0  LIBRARY. 0  ... 

TINYCOMP  depends  on  CODE  GEN.  0.  PARSE  R.O,  and  LIBRARY  .0.  It  is  created  by  relinking  the  system. 


The  Construction  Process 

maki  is  invoked  with  the  following  LIN  IX  command  line  template  (brackets  indicate  optional  fields): 
MAKE  [-f  MAKEFILE ]  [ OPTION  ...  ]  [ TARGET-GRAIN] 

MAKEFILE  Specifics  the  name  of  the  file  containing  the  construction  directives,  if  no  -f  option  is  used 
then  maki:  uses  the  file  named  MAKEFILE  in  die  working  directory. 

OPTION  Specifics  options  like  print  but  do  not  execute  the  command  sequences  or  update  the 

modified  dale  of  the  targets  without  executing  any  command  sequences. 

TARGET-GRAIN  Specifics  the  name  of  the  target  grain  to  be  processed,  if  TARGET-GRAIN  is  not  specified 
then  maki-  will  process  the  first  target  grain  named  in  the  MakcFilc. 

maki:  begins  by  constaicting  a  dependency  graph  from  the  selected  MakcFilc.  F.ach  node  in  the  graph 
corresponds  to  a  grain  mentioned  in  the  MakcFilc.  The  children  of  a  node  represent  die  grains  that  the  grain 
represented  by  the  node  depends  on.  A  request  to  make  a  target  grain  is  processed  by  doing  a  depth-first 
walk  of  the  graph  starting  with  die  node  that  corresponds  to  die  target.  At  each  node  visited,  any  grains  that 
arc  missing  or  whose  children  have  changed  arc  updated. 

maki;  compares  the  creation  dates  of  a  target  grain  and  its  ingredient  grains  as  an  approximate  means  of 
noting  when  changes  occur.  For  instance  if  TARGE T-l  depends  on  INGREDIENT-1  then  maki:  will  assume 
diat  INGREDIENT- 1  has  changed  if  and  only  if  its  creation  date  is  after  the  creation  date  of  TARGET-1. 
Since  UNIX  allows  file  creation  dates  to  be  modified  by  users,  it  is  possible  to  fool  maki-  by  changing  file 
attributes.  However,  since  most  people  do  not  change  file  attributes,  the  makl  mechanism  is  reasonable. 

Without  information  about  how  an  ingredient  has  changed,  make  cannot  determine  whether  a  change  is 
significant  or  not.  Therefore,  make  pessimistically  assumes  that  every  change  to  an  ingredient  grain  will 
effect  the  target  grain,  and  it  will  always  reconstruct  a  target  when  one  of  its  ingredients  has  changed.  Figure 
2-3  contains  the  make  construction  algorithm  written  in  Lisp. 


(DEFUN  MAKE  (NODE) 

(DOLIST  (CHILD  (GET-CHILDREN  NODE)) 

(MAKE  CHILD)) 

(IF  (OR  (NON-EXISTENT-P  NODE)  (CHILDREN-CHANGED-P  NODE)) 
(UPDATE  NODE))) 

(DEFUN  CHILDREN-CHANGED-P  (NODE) 

(<  (CREATION-DATE  NODE) 

(APPLY  #’MAX 

(MAPCAR  #' GET -CREATION -DATE  (GET-CHILDREN  NODE))))) 
Figure  2-3:  make:  Construction  Algorithm 


An  Extended  Kxample  ••  LINT 

The  l  is  i  system  [Johnson  78b|  is  presented  as  an  extended  example  of  using  MAKl .  I  IN  !  examines  C 
source  programs  and  detects  bugs  that  most  C  compilers  cannot.  It  is  also  sensitive  to  constructs  that  arc  legal 
but  may  not  be  portable. 

i  i\l  consists  of  a  UNIX  shell  script  driver,  a  set  of  uni  Library  files,  and  two  C  programs.  Before 
programs  arc  processed  by  die  first  C  program  (i.c..  die  first  pass  of  lint),  Uicy  are  processed  by  the  C 
pre-processor,  which  handles  macro  expansion  and  some  compiler  directives. 

Alter  being  processed  by  the  C  pre-processor,  programs  are  sent  to  the  first  pass  of  UNI.  This  pass  docs 
lexical  analysis  on  the  input  text,  constructs  and  maintains  symbol  tables,  and  builds  trees  for  expressions.  An 
intermediate  file  that  consists  of  lines  of  ASCII  text  is  produced.  Kach  line  contains  an  external  identifier 
name,  an  encoding  of  the  context  in  which  it  was  seen  (use.  definition,  declaration,  etc.),  a  type  specifier,  and  a 
source  file  name  and  line  number,  flic  information  about  variables  local  to  a  function  or  file  is  collected  by 
accessing  the  symbol  table,  and  examining  the  expression  trees.  Comments  about  local  problems  are 
produced  as  detected.  The  information  about  external  names  is  collected  in  die  intermediate  file. 

I  ini  libraries  arc  collections  of  definitions  of  external  names  that  arc  appended  to  the  intermediate  file 
generated  by  the  first  pass  of  I  INI.  They  arc  used  to  provide  I  INI  with  a  set  of  definitions  for  commonly  used 
external  names  without  processing  the  source  that  contains  die  definitions,  flic  most  commonly  used 
libraries  contain  the  definitions  for  die  functions  that  arc  supplied  by  the  UNIX  C  run  time  environment. 
Users  can  create  their  own  libraries  of  commonly  used  names  in  order  to  alleviate  repeated  processing. 

After  all  the  source  files  and  library  descriptions  have  been  collected,  the  intermediate  file  is  sorted  to 
bring  all  information  collected  about  a  given  external  name  together.  The  second  pass  of  LINT  then  reads  the 
lines  from  the  intermediate  file  and  compares  all  of  the  definitions,  declarations,  and  uses  for  consistency. 

f  igure  2-4  contains  the  Makef  ile  for  I  IN  I  .  The  primary  point  of  this  example  is  diat  Makefiles,  even  for 
medium  sized  systems  like  un  i.  arc  difficult  to  understand.  The  BUILD  description  mechanism  introduced  in 
chapter  3  provides  a  much  simpler  way  to  describe  systems. 

The  first  part  of  the  I  IN  f  Makefile  contains  macro  definitions.  These  definitions  are  used  to  specify 
directories  (c.g.,  M),  compilation  flags  (c.g.,  CFLAGS),  and  to  group  files  (c.g..  LINTLIBS).  The  target  ALL  is 
used  to  name  the  major  subsystems  of  the  LINT.  The  next  cluster  of  specifications  manages  the  first  pass  of 
LINT.  There  is  an  entry  for  each  library  file  provided  with  LINT.  Kach  of  these  specifics  that  a  LINT  library  file 
is  dependent  upon  a  library  source  file  and  the  first  pass  of  LINT.  Libraries  depend  on  the  first  pass  of  LINT 
because  they  arc  constructed  by  it.  The  targets  that  specify  management  for  the  second  pass  of  UNT  are 
LPASS2  and  LPASS2 . 0. 

The  LI  NT  ALL,  INSTALL,  SHRINK,  and  CLEAN  targets  arc  not  grains  at  all.  rather,  they  are  used  to 
initiate  installation  and  removal  of  LINT.  A  request  to  make  any  of  these  will  always  result  in  the  associated 
command  sequence  being  executed  because  the  corresponding  files  do  not  exist  in  the  UNIX  environment 
The  use  of  non-existing  grains  to  force  command  sequences  to  be  executed  is  a  popular  and  useful  feature  of 
makt.  The  functionality  provided  by  these  target  grains  is  an  example  of  how  construction  tools  can  be  used 
for  more  than  just  system  construction. 


M=/USR/SRC/LIB/MIP 
Cf l ACS= -0  -DFLEXNAMES 

L1NTI IBS=LLIB-P0RT.LN  LL1B-LC . LN  LUB-LM.LN  LLIB-LMP. LN  LL IB-LCURSES. LM 
AIL:  LPASS1  LPASS2  S(LINTLIBS) 

LPASS1 :  CGRAM.O  XDffS.O  SCAN.O  C0MM1.0  PFIN.O  TREES. 0  OPTIM.O  UNI  .0  HASH.O 

CC  CGRAM.O  XDEfS.O  SCAN.O  C0MM1.0  PFTN.O  TREES. 0  OPTIM.O  UNT  O  HASH.O  -0  LPASS1 

TREES. 0:  $(M)/MANII EST  MACDEIS  S(M)/MFILE1  S(M)/TREES.C 
CC  -C  S(CHAGS)  -IS(M)  -1.  S(M) /TREES . C 
OPTIM.O:  $(M)/MANIFFST  MACOl FS  $(M)/MFIL£1  $(M)/OPTIM.C 
CC  -C  S(CFIAGS)  -I$(M)  -I.  ${M)/OPTIM.C 
PFTN.O:  $(M)/MANI F E ST  MACOEFS  $(M)/MFILE1  S(M)/PFTN.C 
CC  -C  S(CFIACS)  —  1 S ( M )  -1.  $(M)/PFTN.C 
UNT  O:  $(M)/MANIFEST  MACOEFS  $(M)/MFILE1  IMANIFEST 
CC  -C  $( Cf LAGS)  - 1 S ( M )  -I.  L1NT.C 
SCAN.O:  $(M)/MAN1FEST  MACOETS  S(M)/MFILE1  S(M)/SCAN.C 
CC  -C  S( CF LAGS )  -I$(M)  -I.  S(M)/SCAN.C 
XDFFS.O:  $(M) /MANIFEST  $(M)/MF1LE1  MACOEFS  S(M)/XDEFS.C 
CC  -C  $( CF LAGS )  - l$(M)  -I.  S(M)/XOEFS.C 
COMM  1.0:  S(M)/MAN  I F  E  SI  S(M)/MUIE1  S(M)/COMMON  MACOEFS  S(M)/C0MM1.C 
CC  -C  S(CFLAGS)  -I.  -  I S ( M )  S(M)/C0MM1.C 
CGRAM.O:  $(M)/MANIFLST  S(M)/MU1E1  MACDEFS  CGRAM.C 
CC  -C  S(CFLAGS)  - IS(M)  -I.  CGRAM.C 

CGRAM.C:  $(M)/CGRAM.Y 

YACC  J(M)/CGRAM. Y 
MV  Y.TAB.C  CGRAM.C 

LUB-PORT  .  LN:  LLIB-PORT  LPASSI 

-(/LIB/CPP  -C  -DUNT  LUB-PORT  |  . /LPASSI  -PUV  >  LLIB-PORT. LN  ) 

LUB-LM.LN:  LLIB-LM  LPASSI 

-(/LIB/CPP  -C  -0L1HT  LUB-LM  |  ./LPASSI  -PUV  >  LUB-LM.LN  > 

LLIB-LMP. LN.  LLIB-LMP  LPASSI 

-(/LIB/CPP  -C  -DUNT  LLIB-LMP  |  ./LPASSI  -PUV  >  LLIB-LMP. LN  ) 

LLIB-LC . IN :  LLIB-LC  LPASSI 

-(/LIB/CPP  -C  -OLINT  LLIB-LC  |  ./LPASSI  -V  >  LLIB-LC. LN  ) 

LLIB-LCURSES . LN :  LLIB-LCURSES  LPASSI 

-(/LIB/CPP  -C  -DLINT  LLIB-LCURSES  |  ./LPASSI  -V  >  LLIB-LCURSES. LN  ) 

LPASS2:  LPASS2.0  HASH.O 

CC  LPASS2.0  HASH.O  -0  LPASS2 

LPASS2.0:  S(M)/MANIFEST  LMANIFEST 

CC  S(CFLAGS)  -C  -IS(M)  -I.  LPASS2.C 

LINTALL: 

LINT  -HPV  -I.  -I$(M)  S(M)/CGRAM. C  S(M)/XDEFS.C  S(M)/SCAN.C  \ 

$(M)/PFTN.C  $(M)/TR£ES . C  S(M)/OPTIM.C  LINT.C 
INSTALL:  ALL  SHELL 

INSTALL  -S  LPASSI  /USR/LIB/LINT/LINT1 
INSTALL  -S  LPASS2  /USR/LIB/LINT/LINT2 

FOR  I  IN  LLIB-* ;  DO  INSTALL  -C  -M  644  SSI  /USR/LIB/LINT ;  DONE 
INSTALL  -C  SHELL  /USR/BIN/LINT 

SHRINK: 

RM  -F  *.0 
CLEAN:  SHRINK 

RM  -F  LPASSI  LPASS2  CGRAM.C  S(LINTLIBS) 

Figure  2-4:  MakeFile  For  UNT 


Deficiencies 

Phrased  in  terms  of  construction.  The  fundamental  problem  with  make  is  that  it  forces  users  to 
manipulate  lists  of  construction  directives.  People  do  not  normally  think  about  systems  in  terms  of  the  steps 
used  to  construct  them,  and  therefore  these  lists  are  difficult  to  understand.  MAKI  should  present  a  more 
natural  user  interface  and  then  work  from  the  user  supplied  information  towards  the  construction  information 
that  it  requires. 

MAKI  does  not  include  an  adequate  means  for  saving  and  reusing  common  construction  patterns.  The 
introduction  of  such  a  facility  would  shorten  Makefiles  since  common  patterns  would  be  replaced  with  single 
identifiers.  The  definition  of  the  identifier  would  document  and  highlight  die  intended  construction  pattern. 
The  functionality  described  in  diis  paragraph  is  usually  provided  by  a  macro  mechanism,  however  the  MAKE 
macro  facility  is  too  simple  -  it  docs  not  even  allow  for  parameterized  macros. 

No  underlying  task  descriptions.  Systems  dial  keep  knowledge  about  construction  separate  from 
knowledge  about  systems  can  be  extended  by  adding  to  the  construction  knowledge  without  altering  existing 
system  models.  Pitman  (Pitman  84|  discusses  the  importance  of  separating  know  ledge  about  systems  from 
knowledge  about  construction  tasks.  MAKE  docs  not  use  task  descriptions  at  all  and  cannot  be  extended 
without  changing  existing  Makefiles. 

Intermediate  grains  are  referenced.  Maintained  can  only  change  systems  by  manipulating  source  grains  or 
requesting  that  goal  grains  be  constructed.  Maintained  do  not  manipulate  intermediate  grains  and  it  would 
be  nice  if  these  grains  did  not  need  to  appear  in  Makefiles. 

All  source  grains  need  not  he  referenced.  MAKE  allows  system  descriptions  to  omit  source  p  ins  that  arc 
also  goal  grains  since  there  is  no  command  sequence  that  uses  or  effects  them,  for  example,  there  is  nothing 
that  forces  UNIX  Shell  Scripts  to  be  included  in  Makefiles.  The  absence  of  references  to  Shell  Scripts  would 
be  a  serious  omission  if  someone  were  using  a  Makefile  to  determine  which  grains  needed  to  be  copied  when 
transporting  a  system. 


2.2  DEFSYSTEM 

Dl-I  SYSl  l  M  [Weinrcb  and  Moon  81]  is  a  construction  directive  based  tool  that  is  used  to  install  and 
maintain  l.isp  Machine  software.  The  Dl  l  SYSl  l  M  analog  to  MAKI  ’s  Makef  ile  is  called  a  system  description 
DllSYSllM  system  descriptions  contain  a  mixture  of  system  modeling  information  and  construction 
directives.  DllSYSllM  requires  that  command  sequences  (called  transformations)  be  formally  defined  before 
they  arc  used;  this  is  different  from  the  maki-  approach  of  allowing  unlimited  use  of  UNIX  command 
sequences. 

System  descriptions  arc  made  by  DEFSYSTEM  macro.  Calls  to  DEFSYSTEM  have  the  form: 

(DEFSYSTEM  SYSTEM-NAME 
( KEYWORD  ARGS  ...) 

( KEYWORD  ARGS  ...) 

...) 

The  options  selected  by  the  keywords  fall  into  two  general  categories:  properties  of  the  system  and 
transformations. 

There  arc  three  main  Di-ISYSTI-M  property  keywords: 

:  NAME  Specifics  a  "pretty"  version  of  SYSTEM-NAME  for  use  in  printing. 

:  PATHNAME-DEFAULT 

Specifics  a  local  default  within  the  definition  of  the  system  for  strings  to  be  parsed  into 
pathnames. 

:  MODULE  Assigns  a  name  to  a  group  of  files  within  the  system. 

A  transformation  is  an  operation,  such  as  compiling  or  loading,  that  takes  one  or  more  files  and  performs 
some  operation  on  them.  There  are  two  types  of  DF.FSYSTHM  transformations:  simple  and  complex.  A  simple 
transformation  is  a  single  operation  on  a  module,  such  as  compiling  it  or  loading  it.  A  complex 
transformation  combines  several  transformations;  for  example,  compiling  and  then  loading  the  results  of  the 
compilation. 

The  general  format  of  a  simple  transformation  is: 

(NAME  INPUT  PRE-CONDITIONS ) 

NAME  The  name  of  the  transformation  to  be  performed  on  the  files  specified  by  INPUT. 

Examples  of  transformation  names  arc  :FASL0AD  and  :COMPILE-LOAD-INIT  (these 
transformations  are  described  below). 

INPUT  A  module  or  nested  transformation. 

PRE-CONDITIONS 

Optional.  Specifics  transformations  that  must  occur  before  the  current  transformation 
itself  can  take  place.  The  format  is  either  a  list  ( NAME  MODULE-NAMES  . . .  ).  or  a  list  of 
such  lists.  Each  of  these  lists  declares  that  the  transformation  NAME  must  be  performed  on 
the  named  modules  before  the  current  transformation  can  take  place.  (The  Lisp  Machine 
documentation  calls  pre-conditions  i1ci>endencies.) 


1  lie  following  simple  transformations  are  pre-defined: 


:FASLOAD  l.oads  the  indicated  file  when  a  newer  version  of  the  file  exists  than  was  read  into  the 
current  environment. 

:  COMPILE  Compiles  the  indicated  file  when  the  source  file  has  been  been  updated  since  the  compiled 
code  file  was  written. 

Unlike  simple  transformations,  complex  transformations  do  not  have  any  standard  form.  ITic  pre-defined 
complex  transformations  arc: 

:COMPILE-LOAD 

Compiles  and  then  loads  the  input  files.  It  has  the  form: 

( :COMPILE-LOAD  INPUT  COMPILE-CONDITIONS  LOAD-CONDITIONS) 
and  is  exactly  die  same  as 

( : FASLOAD  (rCOMPILE  INPUT  COMPILE-CONDITIONS)  LOAD-CONDITIONS) 

: COMPILE -LOAD- IN IT 

Compiles  and  loads  the  input  files.  This  transformation  is  sensitive  to  changes  made  to  an  additional 
dependency  list.  It  has  the  form: 

( :COMPILE-LOAD-INIT  INPUT  ADDITIONAL-DEPENDENCIES 
COMPILE-PRE-CONDITIONS  LOAD-PRE-CONDITIONS) 

INPUT  will  be  compiled  and  loaded  whenever  its  source  file  or  any  of  the  modules  listed  in 
ADDITIONAL-DEPENDENCIES  are  updated.  Note,  the  ADDITIONAL-DEPENDENCIES  field  of  this 
transformation  specifics  the  same  kind  of  construction  dependency  as  MakeFilc  entries  do. 

It  is  important  to  distinguish  between  transformation  declarations  and  transformation  references. 
Transformations  arc  declared  by  keyword  lists  in  calls  to  DEFSYSTEM.  Transformations  are  referenced  in 
pre-condition  lists.  The  transformations  referenced  in  a  pre-condition  list  must  be  declared  somewhere  in  the 
system  description. 

DLFSYSTEM  contains  a  facility  for  defining  new  transformations.  New  simple  transformations  are  defined 
using  the  DEFINE -SIMPLE -TRANSFORMATION  macro.  Calls  have  the  form: 

(DEFINE -SIMPLE -TRANSFORMATION  NAME  FUNCTION  DEFAULT-CONDITION 

INPUT-FILE-TYPES  OUTPUT-FILE-TYPES ) 

NAME  The  name  of  the  transformation  being  defined. 

FUNCTION  A  function  to  be  called  when  the  transformation  is  performed. 

DEFAULT-CONDITION 

llie  function  that  is  called  in  order  to  determine  if  the  transformation  should  be 
performed. 

INPUT-FILE-TYPES 

Specifics  the  types  of  the  input  files  to  the  transformation.  Lisp  Machine  file  type 
specifications  arc  filename  extensions  (c.g.,  "lisp”  or  "bin”). 

OUTPUT-FILE-TYPES 

Specifics  the  types  of  the  output  files  produced  by  the  transformation. 

For  example,  to  define  a  simple  transformation  called  :  LISP-YACC  that  calls  LISP-YACC  to  derive  parsers 
written  in  Lisp  from  HNF  grammars,  the  following  definition  could  be  made.  (If  a  utility  like  YACC  were 


desired  on  the  Lisp  Maehine  it  would  probably  be  implemented  with  a  macro  and  not  a  separate  parser 
generating  tool.) 

(DEFINE -SIMPLE -TRANSFORMAT ION  :LISP-YACC  #'LISP-YACC 
# ' F I LE -NEWER- THAN- F ILE-P  (: GRAMMAR)  (  : LISP) ) 

LISP-YACC  will  be  invoked  whenever  the  input  file  (i.e..  the  grammar)  is  newer  than  the  output  file  (i.c.,  the 
parser).  In  other  words,  die  transformation  will  be  performed  whenever  the  source  file  is  updated.  Notice 
that  this  transformation  relies  on  grain  creation  dates  in  exactly  the  same  way  that  maki  docs. 

Complex  transformations  arc  defined  as  Lisp  macros.  Here  is  the  definition  of  the  :  COMPILE-LOAD 
transformation  that  was  described  earlier: 

(DEFMACRO  (: COMPILE-LOAD  DEF SYSTEM-MACRO) 

(INPUT  &OPTIONAL  COMPILE-PRE-CONDITIONS  LOAD-PRE-CONDITIONS) 

* ( : FASLOAD  (rCOMPILE  .INPUT  .COMPILE-PRE-CONDITIONS) 

.LOAD-PRE-CONDITIONS)) 

A  Small  Example  -  TINYCOMP 

Figure  2-5  contains  die  DFISYSTEM  description  fora  Lisp  implementation  ofTtNYCOMP. 

(DEFSYSTEM  TINYCOMP 

( : MODULE  DEFS  "DEFINITIONS") 

( : MODULE  PARSER  "PARSER") 

(:MODULE  CODE-GENERATOR  "CODEGEN") 

( : MODULE  LIBRARY  "LIBRARY") 

(: FASLOAD  DEFS) 

(: FASLOAD  LIBRARY) 

( :COMPILE-LOAD-INIT  CODE-GENERATOR  (DEFS)  ( : FASLOAD  DEFS)) 

( :COMPILE-LOAD-INIT  (:LISP-YACC  PARSER)  (DEFS)  ( : FASLOAD  DEFS))) 

Figure  2-5:  defsystem  Description  For  tinycomp 

The  tinycomp  description  contains  a  set  of  module  definitions  followed  by  a  series  of  transformations. 
The  transformations  in  the  description  have  the  following  interpretation: 

(: FASLOAD  DEFS) 

Specifics  that  DEFS  should  be  loaded  whenever  it  is  updated.  There  are  no  pre-conditions  to  be  satisfied 
before  the  loading  can  take  place. 

(: FASLOAD  LIBRARY) 

Specifics  that  LIBRARY  should  be  loaded  whenever  it  is  updated.  There  are  no  pre-conditions  to  be 
satisfied  before  the  loading  can  take  place. 

( :COMPILE-LOAD-INIT  CODE-GENERATOR  (DEFS)  ( : FASLOAD  DEFS)) 

Specifics  that  CODE -GENERATOR  should  be  be  compiled  and  loaded  whenever  it  or  DEFS  changes.  Before 
the  compilation  can  take  place.  DEFS  must  be  loaded. 

( :COMPILE-LOAD-INIT  (:LISP-YACC  PARSER)  (DEFS)  ( : FASLOAD  DEFS)) 

Specifics  that  a  parser  derived  from  PARSER  is  to  be  compiled  and  loaded.  A  new  parser  is  produced 
whenever  PARSER  changes.  The  compiler  and  loader  arc  invoked  whenever  DEFS  or  the  derived  parser 
changes.  :  LISP-YACC  will  not  be  invoked  if  only  DEFS  changes.  Prior  to  compilation,  DEFS  must  be 
loaded. 


The  Construction  Process 

Systems  previously  modeled  with  DEFSYSTEM  arc  constructed  by  calling  MAKE-SYSTEM.  Calls  have  the 
form: 

(MAKE-SYSTEM  SYSTEM-NAME  &REST  OPTIONS) 

SYSTEM-NAME  Specifies  a  system  previously  modeled  with  DEFSYSTEM. 

OPTIONS  Specifies  options  like  print  the  transformations  that  would  be  done  but  don't  do  them  and  so 

forth. 

The  construction  dependency  graph  specified  by  the  transformations  and  pre-conditions  in  the 
Dl  l  system  description  of  SYSTEM-NAME  is  analyzed  in  order  to  determine  what  construction  needs  to  be 
done.  Kach  transformation  is  applied  by  first  applying  any  transformations  referenced  as  pre-conditions,  and 
then  updating  the  input  module  if  it,  or  any  modules  listed  in  additional  dependency  lists,  have  been  changed. 
Notice  that  the  transformation  applications  arc  ordered  by  the  pre-condition  lists. 

l  ike  make.  defsYSTFM  uses  simple  functions  based  on  file  creation  dates  in  order  to  determine  when  a 
module  should  be  reconstructed.  However,  unlike  maki:,  dm  system  allows  the  optional  specification  of 
predicates  that  control  when  construction  is  done.  The  new  predicates  can  replace  the  simple  ones  that  arc 
supplied  with  DEISYSTEM. 

DEFSYSTEM  includes  a  patching  facility.  It  allows  small  changes  to  be  made  to  a  system  without  invoking 
the  DEISYSTEM  transformation/dcpcndcncy  mechanism.  Kach  set  of  changes  is  stored  in  a  patch  file  that 
typically  contains  new  function  definitions  or  redefinitions  of  old  functions.  Kach  patch  is  assigned  a  number. 
If  a  system  contains  patches,  then  the  patches  arc  loaded,  in  order,  after  the  unpatched  version  of  the  system  is 
loaded. 

An  Extended  Example  -  LINT 

The  DEISYSTEM  description  for  a  Lisp  implementation  of  LINT  is  presented  in  figure  2-6.  Although  the 
DEFSYSTEM  description  is  easier  to  understand  than  the  corresponding  MakeFile  (figure  2-4),  it  is  still  difficult 
to  understand. 

The  :  BUILD- LI  NT -LIBRARY  transformation  is  assumed  to  have  been  defined  and  has  the  form: 

( :BUILD-LINT-LIBRARY  INPUT  PRE-CONDITIONS ) 

It  constructs  LINT  library  files  from  LINT  library  sources.  The  transformation  allows  the  optional  specification 
of  pre-conditions,  and  is  applied  if  either  INPUT,  or  the  first  pass  of  LINT  is  updated. 

The  first  keyword  form  in  the  lint  deesysit-m  description  specifies  a  system-wide  default  directory.  The 
next  block  of  keyword  forms  declare  the  various  modules  which  comprise  lint.  The  final  block  of  forms 
declare  the  transformations  used  to  construct  LINT.  Notice  that  as  transformations  arc  nested  and  pre¬ 
conditions  are  added,  the  transformation  declarations  become  increasingly  difficult  to  understand. 

Deficiencies 

Phrased  in  terms  of  construction.  Like  MAKE,  DEFSYSTEM  is  a  construction  directive  based  tool.  This  is 
the  primary  reason  that  Dl  l  syst  em  descriptions,  although  easier  to  understand  than  Makefiles,  arc  still 
awkward. 

One  reason  that  dffsystem  descriptions  are  easier  to  understand  than  Makefiles  is  because  dee'SYSP'M 
is  not  purely  construction  directive  based,  DEI  system  's  :  MODULE  declarations  allow  for  the  logical  grouping 
of  grains  into  higher  level  modules.  This  grouping  abstraeLs  away  from  low  level  construction  information, 
and  provides  a  more  natural  way  for  users  to  describe  systems  than  make  docs. 

DEFSYSTEM  supports  the  sharing  of  common  construction  patterns  through  the  declaration  of 
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(OEFSYSTEM  LINT 

( : PATHNAME-DEFAULT  "/USR/SRC/LIB/MIP" ) 

( : MODULE  DEFINITIONS-1  ( "MACDEFS"  "MANIFEST"  "MFILE1"  "LMANIFEST” ) ) 

( : MODULE  DEFINITIONS-2  ("MANIFEST"  "LMANIFEST")) 

(: MODULE  PARSER  "CGRAM" ) 

( : MODULE  PASS1  ( "XDEFS"  "SCAN"  "COMM1"  "PFTN"  "TREES"  "OPTIM" 

"LINT"  "HASH")) 

( : MODULE  PASS2  ( "LPASS2"  "HASH")) 

( : MODULE  DRIVER  "SHELL") 

( : MODULE  LIBRARIES  ( "LLIB-PORT"  "LLIB-LC"  "LLIB-LM"  "LLIB-LMP" 

"LLIB-LCURSES" ) ) 

( : FASLOAD  DEFINITIONS-1) 

( : FASLOAD  DEFINITIONS-2) 

( :COMPILE-LOAD  DRIVER) 

( :COMPILE-LOAD-INIT  PASS1  (DEFINITIONS-1)  (: FASLOAD  DEFINITIONS-1)) 

( :COMPILE-LOAD-INIT  PASS2  (DEFINITIONS-2)  ( : FASLOAD  DEFINITIONS-2)) 

( :COMPILE-LOAD-INIT  (:LISP-YACC  PARSER)  (DEFINITIONS-1) 

( : FASLOAD  DEFINITIONS-1)) 

( :BUILD-LINT-LIBRARY  LIBRARIES  (: FASLOAD  DRIVER  PARSER  PASS1))) 

Figure  2-6:  defsysti-m  Description  For  ijnt 

transformations.  This  makes  DEI^SYSTEM  system  descriptions  easier  to  produce  and  understand  than 
MakeFiles.  However,  since  it  is  possible  to  avoid  the  declaration  of  a  complex  transformation  by  using  nested 
transformations,  DHI'SYSTEM  still  allows  for  common  patterns  to  be  repeated  instead  of  shared. 

No  underlying  task  descriptions.  Although  Dl-FSYSTEM  has  embedded  knowledge  about  Lisp  compilation 
and  loading  it  docs  not  include  a  mechanism  -for  describing  construction  tasks  and  therefore  cannot  be 
extended  without  great  difficulty. 

Intermediate  grains  arc  referenced.  DFFSYSTFM  does  not  differentiate  between  source,  intermediate,  and 
goal  grains.  In  general,  intermediate  grains  are  hidden  by  complex  transformations.  For  example,  there  are 
no  references  to  intermediate  grains  in  figures  2-5  and  2-6.  While  defsystem  docs  not  force  intermediate 
grains  to  be  included,  it  docs  not  prohibit  them  either. 

All  source  grains  need  not  be  referenced.  In  a  I.isp  environment,  nothing  can  be  used  before  it  is  loaded. 
This  means  that  any  grain  that  participates  in  a  Lisp  system  will  be  involved  in  some  construction,  and 
therefore,  it  is  not  as  natural  to  omit  a  source  grain  from  a  defsystem  description  as  it  is  to  omit  one  from  a 
MakcFilc.  This  difference  between  make  and  defsystem  comes  from  differences  between  the  UNIX  and 
Lisp  environments,  and  not  from  important  differences  between  the  two  tools. 

2.3  Other  Tools 

DeRemer  and  Kron  introduced  the  terms  programming-in-thc-largc  and  programming-in-the-small 
[DeRemer  and  Kron  76]  to  distinguish  between  the  writing  of  modules  and  the  structuring  of  modules  into 
systems.  Consistent  construction  is  just  one  programming-in-thc-largc  issue,  others  include  source  code 
management,  module  interconnection  specification.  and  version  control.  A  brief  summary  of  these  other 
issues  and  projects  that  focus  upon  them  is  presented  here  for  completeness.  The  consistent  construction 
components  of  these  projects  do  not  differ  from  make  or  defsysh  M  in  any  significant  way. 

When  several  people  arc  working  on  a  system  simultaneously,  it  is  important  to  regulate  access  to  the 
source  code  modules  in  order  to  ensure  that  someone  docs  not  attempt  to  modify  a  module  while  someone 
else  is  modifying  that  same  module.  A  common  scheme  is  to  implement  a  librarian  that  regulates  access  to 
system  components  via  a  chcck-in/chcck-oul  mechanism.  In  short,  only  one  person  is  allowed  to  check-out  a 
module  for  update  at  any  tin,  \  Anyone  can  read  a  module  at  any  time.  Source  code  management  systems 
me  ucscnacd  m  the  following  papeis |Koelil.m  i  .a.  t  li-.ui  u.  et.  al.  ME  1  .o.-.k  ,  .mu  I  }  a«.h  I  ewisST). 


All  of  the  problems  mentioned  above  arc  compounded  if  the  programming  environment  is  distributed 
over  a  network.  Schmidt  addresses  these  issues  |Schmidt  82], 

It  is  often  the  ease  that  there  arc  families  of  systems  being  managed.  For  example  there  may  be  several 
public  releases  of  a  system,  internal  releases,  experimental  versions  and  so  on.  It  is  also  common  for  there  to 
be  several  versions  of  a  system  intended  to  run  on  different  hardware  configurations.  Kach  member  of  a 
family  of  software  systems  usually  shares  many  components  with  other  members  of  the  family.  Maintainers 
of  such  families  need  to  worry  about  which  versions  of  which  modules  arc  used  in  each  member  of  the  family. 
Tichy  and  Coopridcr  attacked  the  problems  associated  with  the  representation  and  management  of  software 
families  [Coopridcr  79,  Tichy  80,  Tichy  841. 


3.  The  BUILD  Reference  Level 


This  chapter  introduces  build's  reference  based  system  modeling  scheme.  BUILD  system  models  are  very 
easy  to  interpret  because  they  contain  nothing  more  titan  declarations  of  how  grains  arc  grouped  to  form 
modules  and  how  these  modules  refer  to  each  other.  Although  they  do  not  present  any  construction 
dependencies  explicitly,  they  can  be  used  to  derive  all  of  the  construction  information  found  in  construction 
based  models  (sec  Chapter  5).  Construction  models  cannot  be  used  to  derive  the  reference  information  found 
in  reference  models.  Reference  models  arc  far  less  confusing  than  the  construction  based  models  because 
they  arc  written  in  a  language  that  replaces  low  level  grain  construction  information  with  higher  level  inter¬ 
module  reference  patterns. 

3.1  Modules 

It  is  often  the  case  that  groups  of  grains  arc  conceived  as  one  logical  entity  but  arc  split  up  (c.g.  into  files) 
for  other  reasons.  Modeling  schemes  that  represent  systems  only  at  the  level  of  the  individual  grain  do  not 
have  the  ability  to  express  this  kind  of  grouping.  1’hc  module  construct  used  by  BUILD  (and  DEI^YSTEM) 
allows  these  groupings  to  be  made  explicitly  in  system  descriptions. 

BUII.D  module  declarations  have  the  form: 

( : MODULE  MODULE-NAME  GRAIN-TYPE  &REST  GRAINS) 

MODULE-NAME  The  name  of  a  module.  The  name  must  be  unique  within  the  system  model. 

GRAIN-TYPE  The  name  of  a  grain  type  recognized  by  BUILD.  Each  grain  is  assumed  to  be  an  instance  of 
this  type. 

GRA I  NS  The  names  of  the  grains  that  comprise  the  module. 

The  following  form  declares  that  MA  IN  is  a  Lisp  source  module  composed  of  the  single  grain  MAIN .  L I SP, 
(:MODULE  MAIN  :LISP-SOURCE  "MAIN. LISP") 
and  the  form: 

( : MODULE  DEFS  :C-SOURCE  "DEFINITIONS- 1 . C"  "DEF INIT IONS-2 . C"  ) 
declares  that  DEFS  is  a  C  source  module  with  two  grains  named  DEFINITIONS-1  .C  and 
DEFINITI0NS-2.C. 

build  can  use  grain  type  information  without  considering  module  references  to  determine  a  great  deal 
about  the  construction  of  grains.  For  instance,  build  knows  how  to  invoke  the  correct  compiler  on  C  or  Lisp 
source  files  or  how  to  construct  lint  library  files  from  library  sources  by  utilizing  grain  type  information 
alone. 

3.2  References 

build  infers  construction  dependencies  from  reference  assertions  by  taking  advantage  of  the  fact  that 
construction  dependencies  arc  caused  by  references  between  modules.  If  two  modules  do  not  refer  to  each 
other,  then  it  is  impossible  for  there  to  be  a  construction  dependency  that  involves  them.  When  the  assertion 
is  made  that  module f  refers  to  module ?  build  pessimistically  assumes  that  each  grain  in  module f  refers  to  each 
grain  in  module r 

References  with  the  same  name  may  be  handled  differently  depending  upon  the  grain  types  of  the 
modules  involved  in  the  reference.  For  instance,  the  calls  reference  between  two  Lisp  source  modules  is 
handled  differently  than  the  c alls  reference  between  two  C  source  modules. 


ill  il  l)  reference  decimations  provide  for  the  specification  of  references  between  modules.  No  meaning  is 
attached  to  die  ordering  of  reference  declarations.  Reference  declarations  have  the  form: 

(REFERENCE  LEFT-ELEMENT  RIGHT-ELEMENT) 

REFERENCE  The  name  of  a  reference  recogni/.cd  by  HUll  I). 

LEFT-ELEMENT  A  module  name  or  list  of  module  names.  All  module  names  used  in  a  reference 
declaration  must  have  been  declared  in  a  module  declaration. 

RIGHT-ELEMENT 

A  module  name  or  list  of  module  names.  All  module  names  used  in  a  reference 
declaration  must  have  been  declared  in  a  module  declaration. 

I'hc  use  of  module  name  lists  as  either  of  the  elements  of  a  reference  declaration  is  syntactic  sugar  that  is 
equivalent  to  the  set  of  reference  declarations  composed  by  enumerating  REFERENCE-NAME  with  each  pair 
in  the  cross  product  of  the  right  and  left  clement  lists.  For  example: 

( : CALLS  (A  B)  (D  E)) 
is  equivalent  to: 

( :CALLS  A  D) 

( : CALLS  A  E) 

( : CALLS  B  D) 

( :CALLS  B  E) 

Here  are  some  reference  triples  and  the  construction  dependencies  that  they  imply: 

( : CALLS  LISP-SOURCE-1  LISP-SOURCE-2) 

Asserts  that  LISP-SOURCE-1  contains  functions  that  call  LISP-SOURCE-2  and  implies  that 
LISP-SOURCE  -2  will  need  to  be  loaded  in  order  for  LISP-SOURCE-1  to  execute. 

( :MACRO-CALLS  LISP-SOURCE-1  LISP-SOURCE-2) 

Asserts  that  LISP-SOURCE-1  uses  macros  defined  in  LISP-SOURCE-2  and  therefore  LISP-SOURCE-2 
must  be  loaded  in  order  for  LISP-SOURCE-1  to  compile  properly.  This  reference  implies  that  if 
LISP-SOURCE-2  changes,  then  LISP-SOURCE-1  will  need  to  be  re-compiled. 

( :CALLS  C-SOURCE-1  C-SOURCE-2) 

Implies  that  the  object  grains  compiled  from  C-SOURCE-2  (as  well  as  the  object  grains  from  any  module 
that  C-SOURCE-2  calls)  need  to  be  linked  into  any  executable  image  that  is  to  include  the  object  grains 
from  C-SOURCE-1. 

( : INCLUDES  C-SOURCE-1  C-SOURCE-2) 

Asserts  that  C-SOURCE-1  contains  the  contents  of  C-SOURCE-2.  This  reference  implies  that  whenever 
the  included  module,  C-SOURCE-2.  changes,  the  including  module,  C-SOURCE-1,  needs  to  be  rebuilt 

HUH  n  uses  triples  (called  reference  signatures)  of  the  form 

CREFERENCE-NAME  LEFT-GRAIN-TYPE-NAME  RIGHT -GRAIN-TYPE-NAME> 
to  identify  references.  HUll  n  uses  grain  type  information  to  distinguish  between  references  that  have  the 
same  name  but  apply  to  different  grain  types.  A  given  implementation  of  BUILD  will  define  the  reference 
signatures  that  arc  commonly  used  in  the  environment  that  HUll  D  is  working  with.  Chapter  5  describes  how 
new  reference  signatures  may  be  added  to  BUll.1). 


3.3  Models 

The  general  form  of  a  nun  l)  system  description  is: 

(DEFMODEL  MODEL-NAME  &REST  DECLARATIONS) 

There  arc  four  kinds  of  declarations  that  may  he  included  in  a  DEFMODEL  form:  module,  reference, 
default  pathname,  and  default  module.  Module  and  reference  declarations  were  described  earlier  in  this 
chapter.  The  default  pathname  declaration  allows  for  the  declaration  of  a  pathname  to  be  used  as  a  template 
for  completing  filenames.  It  has  the  form: 

(.DEFAULT-PATHNAME  PATHNAME) 

The  default  module  declaration  is  used  to  declare  a  module  as  the  default  module  for  nun  D  to  operate  on 
when  construction  requests  for  the  system  are  made.  It  has  the  form: 

( : DEFAULT-MODULE  MODULE-NAME) 

Figure  3-1  contains  the  DEFMODEL  form  for  TiNYCOMl\  The  first  four  declarations  arc  module 
declarations  that  specify  the  grains  and  grain  types  of  the  system  modules.  The  last  three  declarations  specify 
the  references  between  the  modules  in  die  system.  Figure  3-2  contains  the  DEFMODEL  form  for  lINT.  The 
model  is  longer  than  the  TINYCOMP  model  but  no  more  complicated. 

(DEFMODEL  TINYCOMP 

( : MODULE  DEFS  :C-SOURCE  "DEFINITIONS" ) 

(: MODULE  PARSER  : YACC-GRAMMAR  "PARSER") 

( : MODULE  CODE-GENERATOR  :C-SOURCE  "CODEGEN") 

( : MODULE  LIBRARY  :C-OBJECT  "LIBRARY") 

( : INCLUDES  (PARSER  CODE-GENERATOR)  DEFS) 

(: CALLS  PARSER  (LIBRARY  CODE-GENERATOR)) 

( : CALLS  CODE-GENERATOR  LIBRARY)) 

Figure  3-1:  build  Model  For  tinycomp 


(DEFMODEL  LINT 

( :DEFAULT-PATHNAME  "/USR/SRC/LIB/MIP" ) 

( : MODULE  DEFINITIONS-1  :C-S0URCE 
"MACDEFS"  "MANIFEST"  "MFILE1"  "LMANIFEST" ) 

( : MODULE  DEFINITIONS-2  :C-S0URCE  "MANIFEST"  "LMANIFEST") 

( : MODULE  PARSER  : GRAMMAR  "CGRAM") 

( : MODULE  PASS-1  :C-SOURCE  "LINT") 

( : MODULE  PASS-2  :C-SOURCE  "LPASS2") 

( : MODULE  SUPPORT-1  :C-SOURCE 
"XDEFS"  "SCAN”  "C0MM1"  "PFTN"  "TREES"  "OPTIM"  "HASH") 

( : MODULE  SUPPORT-2  :C-S0URCE  "HASH") 

( : MODULE  DRIVER  : SHELL-SCRIPT  "SHELL") 

( : MODULE  LIBRARIES  : LINT -LIBRARY-SOURCE 
"LLIB-PORT”  "LLIB-LC"  "LLIB-LM"  "LLIB-LMP"  "LLIB-LCURSES" ) 

(: INCLUDES  PASS-1  DEFINITIONS-1) 

( : INCLUDES  PASS-2  DEFINITIONS-2) 

( : CALLS  DRIVER  (PASS-1  PASS-2  LIBRARIES)) 

( : CALLS  PASS-1  (PARSER  SUPPORT-1)) 

( :CALLS  PASS-2  SUPPORT-2)) 


Figure  3-2:  build  Description  For  LINT 


4.  The  BUILD  Task  Level 


t  his  chapter  describes  die  task  level  representation  of  systems  used  by  null  l).  A  task  level  model  is 
derived  from  the  reference  level  model  for  each  request  that  null  I)  receives.  The  derived  model  is  then  used 
to  handle  the  request.  (The  phrase  task  level  is  used  in  place  of  the  more  specific  phrase  construction  level 
because  HUH  l)  is  used  for  more  than  just  construction.) 

HlJll  O  task  level  models  arc  acyclic  directed  graphs  with  two  kinds  of  nodes:  g- nodes  which  represent 
grains,  and  /t-noties  which  represent  the  processes  used  to  construct  grains.  Leaf  nodes  represent  source 
grains,  and  root  nodes  represent  goal  grains.  The  link  between  grains  and  the  processes  dial  use  diem  is 
modeled  by  linking  the  g-nodcs  representing  grains  to  the  p-nodcs  representing  the  processes  that  use  diem. 

f  igure  4-1  contains  a  portion  of  die  tusk  graph  used  to  represent  die  compilation  of  PARSER. LISP,  a 
grain  from  a  Lisp  implementation  of  iinycomi*.  This  example  assumes  that  PARSER .  LISP  is  a  source  grain 
and  ignores  the  fact  that  in  iinycomi1.  PARSER .  LISP  is  an  intermediate  module  produced  by  LISP-YACC. 
The  ellipses  represent  g-nodcs  and  the  rectangles  represent  p-nodcs.  There  arc  two  source  nodes. 
PARSER  .LISP  and  DE  FS .  L  ISP.  and  a  single  goal  node.  PARSER .  IMAGE. 

Although  the  use  of  an  acyclic  directed  graph  to  represent  task  processing  is  not  unique  (MAKT  and 
dttsys  i  i:M  use  similar  representations)  die  derivation  of  task  graphs  from  reference  models  is  novel. 


Figure  4-1:  Simple  Task  Graph 


4.1  Grain  types 

Grain  type  objects  are  used  to  represent  the  classes  of  grains  used  by  the  environment  that  BUILD  is 
working  with.  They  are  used  to  represent  all  of  the  kinds  of  grains  that  are  manipulated  by  the  underlying 
environment,  whether  they  are  files  or  not.  For  instance,  the  grain  type  :  LISP-IMAGE  is  used  to  represent 
the  objects  that  result  from  loading  files  into  the  Lisp  environment. 


Defining  Grain  Types 

Grain  types  arc  defined  with  DEFINE-GRAIN-TYPE  and  definitions  have  the  form: 

(DEFINE -GRAIN-TYPE  NAME  &OPTIONAL  FILENAME-EXTENSION) 

NAME  ITic  name  of  the  grain  type  being  defined. 

FILENAME-EXTENSION 

The  default  filename  extension  for  grains  of  this  type.  If  this  field  is  null  then  build 
assumes  diat  grains  of  this  type  are  not  files. 
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figure  4-2  contains  the  grain  type  definitions  used  to  model  I  isp  systems.  The  : LISP-SOURCE  and 
.LISP-BINARY  grain  types  correspond  to  tiles  and  hence  their  definitions  include  default  filename 
extensions  (the  I  isp  Machine  uses  keyword  symbols  to  represent  filename  extensions),  flic  :  LISP- IMAGE 
grain  type  is  not  associated  with  tiles  and  therefore  has  no  default  filename  extension. 

(DEF I NE -GRAIN-TYPE  :LISP-SOURCE  :LISP) 

(DEF I NE -GRAIN- TYPE  :LISP-BINARY  :BIN) 

(DEF I NE -GRAIN-TYPE  :LISP-IMAGE) 

figure  4-2:  Grain  Type  Definitions  for  Lisp 

4.2  G-nodes 

G-nodcs  represent  grains  in  task  graphs,  they  contain  the  following  information: 

NAME  The  name  of  the  grain  represented  by  this  g-nodc. 

TYPE  The  grain  type  object  that  the  grain  represented  by  this  g-nodc  is  an  instance  of. 

MODULE  Optional.  The  module  that  includes  the  grain  represented  by  this  g-nodc. 

CREATOR  Optional.  The  p-nodc  that  represents  the  process  that  created  this  g-nodc.  This  field  will 

be  null  if  the  g-nodc  represents  a  source  grain. 

USERS  A  list  of  p-nodcs  that  depend  on  this  g-node  to  fill  an  input  role. 

INGREDIENTS  A  list  that  represents  the  source  grains  used  to  produce  this  g-node.  Each  element  of  the 

list  is  a  pair  containing  the  name  and  creation-date  of  an  ingredient  grain. 

CREATE-DATE  A  time  stamp  that  represents  the  time  and  date  when  the  grain  that  is  represented  by  this 
g-node  was  created. 

4.3  Process  Types 

Process  type  objects  contain  the  information  pertaining  to  classes  of  process  instances  (represented  by 
p-nodcs).  For  example,  the  Lisp  Machine  implementation  of  build  includes  process  type  objects  for  Lisp 
compilation  and  Lisp  binary  file  loading. 

The  grains  that  are  used  and  produced  by  processes  are  partitioned  according  to  the  roles  that  they  play  in 
them.  Grains  that  processes  use  are  said  to  play  input  roles.  Grains  that  arc  produced  by  processes  are  said  to 
play  output  roles. 

Process  type  objects  contain  role  descriptions  for  each  of  their  input  and  output  roles.  Role  descriptions 
contain  the  following  information: 

NAME  The  name  of  the  role.  It  must  be  unique  within  the  process  type  being  defined. 

GRAIN- TYPE  The  grain  type  name  that  grains  filling  this  role  must  have. 

ARITY  Either  : SINGLE  or  :MULTIPLE.  A  role  with  arity  :  SINGLE  can  have  no  more  than  one 

grain  filling  it.  A  role  with  arity  :MULTIPLE  can  have  an  arbitrary  number  of  grains 
filling  it. 

NAME-SOURCE  Optional.  The  name  of  a  role  used  to  help  derive  names  for  grains  that  will  fill  this  role. 


Defining  Process  Types 

Process  types  are  defined  with  DEF I NE  -PROCESS-  TYPE  and  calls  have  the  form: 

(DEFINE-PROCESS-TYPE  NAME  INPUT-SPEC  OUTPUT-SPEC  STREAM-VAR 

DESCRIBE-FORM  &REST  CONSTRUCT-FORMS) 

The  name  of  the  process  type. 

A  list  of  input  role  descriptions  (discussed  above). 

A  list  of  output  role  descriptions. 

A  variable  name  that  will  be  bound  to  the  output  stream  when  DESCRIBE-FCRM  and 
CONSTRUCT-FORMS  arc  evaluated. 

DESCRIBE-FORM 

A  form  to  be  evaluated  in  order  to  describe  the  processing  represented  by  an  instance  of 
this  process  type.  When  the  form  is  evaluated,  each  role-name  will  be  bound  to  the  names 
of  the  grains  playing  die  role.  Also,  the  symbol  named  by  STREAM-VAR  will  be  bound  to 
the  output  stream. 

CONSTRUCT -FORMS 

The  forms  to  be  evaluated  in  order  to  accomplish  the  processing  represented  by  an  instance 
of  the  process  type.  When  these  forms  arc  evaluated  each  of  the  role-names  and  the 
symbol  named  by  STREAM-VAR  will  be  bound  as  mentioned  above. 

Figure  4-3  contains  the  process  type  definitions  for  Lisp  compilation  and  Lisp  binary  loading.  The 
definition  for  : LISP-COMPILE  specifics  that  there  are  two  input  roles.  SOURCE  and  DEFINITIONS,  and  a 
single  output  role,  BINARY.  SOURCE  has  singular  arity  and  must  be  filled  by  a  : LISP-SOURCE  grain. 
DEFINITIONS  has  multiple  arity  and  can  only  be  filled  by  :LISP-IMAGE  grains.  BINARY  has  singular 
arity  and  must  be  filled  by  a  :  L ISP-B I  NARY  grain.  The  describe  form  produces  descriptions  like: 

"COMPILE  PARSER. LISP" 

The  construct  forms  produce  the  grain  playing  the  BINARY  role  by  compiling  the  grain  playing  the  SOURCE 
role.  The  construct  forms  also  cause  a  notification  of  the  compilation  to  be  sent  to  the  output  stream.  The 
notification  looks  like: 

"COMPILING  PARSER. LISP. 5" 

Processes  often  depend  on  grains  not  explicitly  mentioned  in  their  invocations.  For  example,  in  languages 
that  rely  on  objects  to  be  specified  or  loaded  before  objects  that  refer  to  them  can  be  compiled,  the 
compilation  process  type  must  include  a  role  that  is  used  to  capture  that  dependency.  The  role 
DEFINITIONS  is  used  in  :LISP-COMPILE  in  order  to  express  the  need  for  some  things  to  be  defined 
before  a  Lisp  grain  can  be  compiled.  The  link  between  the  g-node  for  DEFS.  IMAGE  and  the  p-nodc 
representing  the  compilation  of  PARSER.  LISP  in  the  task  model  from  figure  4-1  is  an  example  of  such  a 
dependency  being  modeled.  Another  situation  in  which  it  is  necessary  to  model  a  dependency  not  made 
explicitly  in  command  line  invocation  is  for  C  compilation.  The  :C -COMPILE  process  type  has  the  role 
INCLUDE  to  represent  the  dependency  between  a  file  and  the  files  that  it  includes  via  the  C  ^INCLUDE 
mechanism. 


NAME 

INPUT-SPEC 

OUTPUT-SPEC 

STREAM-VAR 


(DEFINE -PROCESS-TYPE  : L ISP-COHPILE 
((SOURCE  : LISP-SOURCE  : SINGLE ) 

(DEFINITIONS  :LISP-IMAGE  : MULTIPLE ) ) 
((BINARY  : LISP-BINARY  : SINGLE  SOURCE)) 
OUTPUT-STREAM 

(FORMAT  OUTPUT-STREAM  "-"/COMPILE  -A" 

(PATHNAME -MINUS- VERS  ION  SOURCE)) 
(FORMAT  OUTPUT-STREAM  " -'/.COMPILING  -A"  SOURCE) 
(COMPILER:COMPILE-FILE  SOURCE  BINARY)) 


SOURCE  INPUT  ROLE 
DEFINITIONS  INPUT  ROLE 
BINARY  OUTPUT  ROLE 
STREAM-VAR 
DESCRIBE-FORM 

; CONSTRUCT -FORMS 


(DEFINE-PROCESS-TYPE  : LISP-LOAD-BIN 
((BINARY  : LISP-BINARY  rSINGLE) 

(DEFINITIONS  :LISP-IMAGE  -.MULTIPLE)) 

((IMAGE  : LISP- IMAGE  :SINGLE  BINARY)) 
OUTPUT-STREAM 

(FORMAT  OUTPUT-STREAM  "-"/LOAD  -A" 

(PATHNAME-MINUS-VERSION  BINARY)) 
(FORMAT  OUTPUT-STREAM  "-"/LOADING  -A"  BINARY) 
(SI : LOAD-BINARY-F ILE  BINARY  NIL  T)) 


BINARY  INPUT  ROLE 
DEFINITIONS  INPUT  ROLE 
IMAGE  OUTPUT  ROLE 
STREAM-VAR 
DESCRIBE-FORM 

; CON STRUCT -FORMS 


Figure  4-3:  Process  Type  Definitions  For  I  .isp 


4.4  P- Nodes 

Filch  p-nodc  represents  a  process  to  be  invoked  on  the  grains  attached  to  its  input  ports  to  produce  the 
grains  attached  to  its  output  ports.  Kach  role  in  a  process  type  is  represented  as  a  port  in  p-nodcs  of  that  type. 
'Die  grain  type  of  each  g-node  attached  to  a  port  must  be  the  same  as  the  grain  type  associated  with  the  role. 
A  description  of  the  processing  represented  by  a  p-nodc  and  the  g-nodcs  attached  to  its  ports  can  be  produced 
by  applying  DESCRIBE-FORM  from  the  p-nodc's  process  type  object  to  the  p-node.  The  processing 
represented  by  the  p-nodc  can  be  done  by  applying  CONSTRUCT-FORMS  from  the  p-node’s  process  type 
object  to  the  p-node. 

Figure  4-4  contains  an  expanded  view  of  the  p-node  used  to  represent  the  compilation  of  PARSER .  LISP 
in  TINYCOMP. 


Figure  4-4:  F.xpandcd  P-Nodc 


4.5  Task  Ciraph  C  onstraints 

Task  graphs  arc  constrained  in  die  following  ways: 

1.  Task  graphs  arc  acyclic.  A  cycle  in  a  graph  would  imply  that  some  grain  was  needed  in  order  to 
construct  itself. 

2.  The  parent  of  a  g-nodc,  if  there  is  one.  must  be  a  p-nodc. 

3.  A  g-nodc  can  have  no  more  than  one  parent. 

4.  A  g-nodc  without  a  parent  represents  a  source  grain. 

5.  The  children  of  a  g-nodc.  if  there  arc  any.  must  be  p-nodcs.  These  nodes  represent  processes  that 
depend  upon  the  grain  represented  by  the  g-nodc. 

6.  A  g-nodc  without  children  represents  a  goal  grain. 

7.  The  children  of  a  p-nodc  must  be  g-nodcs.  These  g-nodcs  represent  grains  derived  by  the  process 
represented  by  the  p-nodc.  Kach  p-nodc  must  have  at  least  one  child. 

In  other  words,  task  graphs  arc  acyclic  graphs  which  begin  with  g-nodcs  that  represent  source  grains  and  end 
with  g-nodcs  that  represent  goal  grains.  Ihc  g-nodcs  arc  separated  by  p-nodcs  that  represent  the  processes 
dial  derive  later  g-nodcs  from  earlier  ones. 

Figures  1-2,  2-1.  and  4-1  are  examples  of  well  formed  task  graphs. 

4.6  The  Construction  Algorithm 

Figure  4-5  contains  the  algorithm  used  by  BUIIJD  to  perform  the  construction  modeled  by  a  task  graph. 
This  algorithm  is  similar  to  the  one  used  by  MAKE  and  DEFSYSTEM  (figure  2-3),  the  primary  difference 
between  the  two  algorithms  is  in  how  they  make  use  of  creation  dates  to  determine  when  construction  is 
necessary.  The  make  algorithm  uses  file  creation  date  ordering  between  input  and  output  grains  in  order  to 
infer  that  an  input  has  changed  (and  therefore  construction  is  triggered).  In  practice  this  method  works, 
however,  it  relies  on  several  assumptions  that  arc  not  necessarily  true. 

make  and  deesystem  assume  that  files  with  the  same  name  but  different  extensions  are  related.  For 
instance,  they  assume  that  MAIN.O  was  created  by  compiling  MAIN.C.  While  this  is  a  reasonable 
assumption,  it  docs  not  have  to  be  true.  Nothing  prevents  users  from  renaming  files  and  therefore,  there  is  no 
guarantee  that  MAIN.O  actually  came  from  MAIN.C. 

If  an  output  grain  contains  a  file  creation  date  that  is  newer  titan  all  of  the  input  grains  used  to  produce  it, 
then  make  and  deesystem  assume  that  the  output  grain  docs  not  need  to  be  rebuilt  However,  there  is  no 
guarantee  that  file  creation  dates  have  not  been  tampered  with. 

HUH  o  docs  not  use  file  creation  date  ordering  to  infer  that  an  object  has  changed.  RL'II  D  compares  a 
grain's  ingredient  list  with  die  ingredient  list  that  would  result  if  the  processing  modeled  by  the  task  graph 
were  done.  If  the  ingredient  lists  match,  then  the  construction  is  not  done. 

The  prototype  implementation  of  Bi  ll  t>  keeps  a  separate  data  file  that  contains  grain  creation  dates  and 
ingredients.  Such  a  file  would  not  be  needed  if  die  underlying  environment  recorded  the  ingredients  used  to 
produce  an  object.  The  Mesa  environment  [Mitchell  79.  Schmidt  82]  keeps  diis  information  and  exploits  it  in 
order  to  determine  when  processing  needs  to  be  done. 


(DEFUN  CONSTRUCT-G-NODE  (G-NODE) 

(COND  ((SOURCE-NODE-P  G-NODE)  T) 

((OR  (NON-EXISTENT  G-NODE)  ( INGREDIENTS-CHANGED  G-NODE)) 
( MAPCAR  0 ' CONST RUCT -G-NODE  (INPUTS  (PARENT  G-NODE))) 
(DO-CONSTRUCTION  (PARENT  G-NODE))))) 

(DEFUN  INGREDIENTS-CHANGED  (G-NODE) 

(NOT  (EQUAL  (INGREDIENTS  G-NODE) 

(DERIVE -INGREDIENTS  G-NODE)))) 

(DEFUN  SOURCE-NODE-P  (G-NODE) 

RETURNS  T  IF  AND  ONLY  IF  G-NODE 
;;  REPRESENTS  A  SOURCE  GRAIN 
) 

(DEFUN  NON-EXISTENT  (G-NODE) 

;;  RETURNS  T  IF  THE  GRAIN  REPRESENTED  BY  G-NODE 
DOES  NOT  EXIST 

) 

(DEFUN  PARENT  (G-NODE) 

;;  RETURN  THE  PARENT  P-NODE  OF  G-NODE 

) 

(DEFUN  INPUTS  (P-NODE) 

;;  RETURN  THE  INPUT  G-NODES  OF  P-NODE 

) 

(DEFUN  DO-CONSTRUCTION  (P-NODE) 

;;  PERFORM  CONSTRUCTION  REPRESENTED  BY  P-NODE 

) 

(DEFUN  INGREDIENTS  (G-NODE) 

;;  RETURN  THE  INGREDIENT  LIST  USED  TO  CONSTRUCT 
;;  THE  EXISTING  VERSION  OF  G-NODE 
) 

(DEFUN  DERIVE -INGREDIENTS  (G-NODE) 

;;  RETURN  THE  INGREDIENT  LIST  THAT  WOULD  RESULT  IF 
;;  A  NEW  VERSION  OF  G-NODE  WERE  CONSTRUCTED 
) 


Figure  4-5:  build  Construction  Algorithm 


5.  Construction  Requests  and  Task  Graph  Derivation 

Alter  a  system  has  been  modeled  with  DEFMODEL.  ItUll  l>  can  be  called  upon  to  handle  construction 
requests  for  it.  Kach  request  has  the  form: 

(BUILD-REQUEST  MODEL  REQUEST  &OPTIONAL  MODULE  ( MODE  : NORMAL)) 

MODEL  The  name  of  a  model  previously  defined  with  DEFMODEL. 

REQUEST  The  name  of  a  request  recognized  by  nun  .!>  (c.g  rCOMPILE,  :LOAD). 

MODULE  'Hie  name  of  a  module  to  operate  upon.  If  this  field  is  not  specified  then  the  default 

module  for  die  system  (as  defined  with  die  : DEFAULT -MODULE  declaration  form)  is  used. 

MODE  Specifics  one  of  several  construction  modes.  Construction  modes  arc  discussed  below. 

The  prototype  implementation  of  nun  n  has  three  construction  modes  that  behave  as  follows: 

:  NORMAL  Describe  all  of  the  construction  to  be  done,  and  then  ask  the  user  if  null  I)  should  perform 

the  construction  just  described. 

:  DESCRIBE  Describe  all  of  the  construction  to  be  done  but  do  not  perform  it. 

:  NO-CONFIRM  Perform  the  required  construction  without  describing  it  first. 

Sample  BUILD  requests  are: 

(BUILD-REQUEST  TINT -COMP  :LOAQ) 

(BUILD-REQUEST  LINT  : LOAD  ORIVER) 

(BUILD-REQUEST  LINT  : LOAD  DRIVER  DESCRIBE) 

Once  a  request  has  been  received,  a  three  step  process  is  executed  for  each  grain  in  the  module  stated  in 
the  request.  This  process  creates  a  task  model  for  the  request  which  is  then  processed  in  the  manner  outlined 
in  chapter  4.  The  three  steps  are: 

1.  Model  the  construction  that  can  be  deduced  from  the  request  without  considering  any  references. 

This  phase  is  called  pre-reference  request  processing. 

2.  Model  the  construction  that  is  implied  by  the  references  that  involve  the  module  associated  with 
the  request.  This  phase  is  called  reference  processing. 

3.  Model  the  construction  that  can  be  deduced  from  the  request  and  the  graph  built  from  the  earlier 
steps.  This  phase  is  called  post- reference  request  processing. 

After  the  post-reference  processing  has  been  completed  the  task  graph  is  complete  and  can  be  used  to  direct 
the  construction  needed  to  handle  the  request. 

Before  the  construction  process  can  be  explained  in  detail  it  is  necessary  to  present  the  functions  used  to 
view  and  manipulate  task  graphs. 


5.1  Viewing  and  Manipulating  Task  Graphs  --  ACCESS 

Consider  the  following  task  graph: 


Starting  at  a  p-nodc.  the  path  to  any  of  the  g-nodes  connected  to  one  of  its  ports  can  be  specified  by 
mentioning  the  name  of  the  port  desired.  In  the  task  graph  above,  starting  at  the  :  LISP-COMPILE  p-nodc, 
the  step  BINARY  leads  to  DE  F  S  .  B I N . 

A  step  from  a  g-nodc  to  a  p-nodc  can  be  described  by  specifying  the  process  type  of  the  connected  p-nodc 
and  the  role  played  by  the  g-nodc  in  the  p-nodc.  In  the  sample  task  graph  above,  the  step  (BINARY 
:  L  ISP-COMPILE  )  slartingatDEFS.BIN  leads  to  the  :  L ISP-COMPILE  p-nodc. 

Paths  arc  formed  by  listing  steps: 

•The  path  ((SOURCE  :  L  ISP-COMPILE )  BINARY)  starting  at  DEFS.LISP  leads  to 
DEFS.BIN. 

*  I  he  path  ((SOURCE  :LISP-COMPILE  )  BINARY  (BINARY  :LISP-LOAD-BIN))  starting 
at  DEFS.  LISP  leads  tothc:LISP-LOAD-BINp-node. 

*  The  path 

((SOURCE  : LISP-COMPILE )  BINARY-  (BINARY  :LISP-LOAD)  IMAGE) 
starting  at  DEFS. LISP  leads  to  DE  FS .  IMAGE. 

*  The  path 

((IMAGE  : LISP-LOAD)  BINARY  (BINARY  : LISP-COMPILE )  SOURCE) 
starting  at  DEFS.  IMAGE  leads  to  DEFS.LISP. 

The  ACCESS  family  of  functions  arc  designed  to  provide  a  straightforward  mechanism  for  both  viewing 
and  manipulating  task  graphs.  These  functions  are  used  heavily  during  the  task  graph  derivation  process. 
There  arc  three  functions,  ACCESS.  ACCESS+,  and  ACCESS*,  each  of  which  is  SETFable.  The  ACCESS 
functions  have  the  form: 

( FUNCTION  NODE  PATH ) 

FUNCTION  ACCESS, ACCESS+, or  ACCESS*. 

NODE  Hither  a  p-nodc  or  a  g-nodc.  This  node  is  used  as  the  root  of  the  path  to  be  traced  by 

ACCESS-FUNCTION. 

PATH  A  list  of  steps  to  be  traced  from  NODE. 

The  functions  behave  in  the  following  manner: 

ACCESS  Traces  PATH  from  NODE  and  returns  the  last  node  encountered.  An  error  is  signalled  if 

any  step  in  PATH  cannot  be  traced.  An  error  is  signalled  if  there  could  be  more  than  one 
node  that  satisfies  the  path  traced. 
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ACCESS+  Traces  PATH  from  NODE  and  returns  a  list  of  nodes  that  satisfy  the  path.  An  emir  is 

signalled  if  any  step  in  PATH  cannot  be  traced. 

ACCESS*  Traces  PATH  from  NODE  and  returns  the  single  node  lltat  satisfies  the  path.  An  error  is 

signalled  if  there  could  be  more  than  one  node  diat  satisfies  PATH.  New  nodes  arc  created 
if  steps  in  PATH  do  not  exist. 

Any  ACCESS  call  that  returns  a  single  node  may  be  used  to  specify  the  root  of  another  call  to  ACCESS,  in 
other  words,  die  following  two  calls  arc  equivalent: 

(ACCESS  NODE  (STEP1  STEP2  STEP3) ) 

(ACCESS  (ACCESS  (ACCESS  NODE  STEP1)  STEP2 )  STEP3) 

Kach  of  the  ACCESS  functions  can  be  SETFed.  Calls  have  the  form: 

(SETF  (ACCESS  ROOT-NODE  PATH)  END-NODE) 

Knsurcs  that  future  calls  to  ACCESS  with  ROOT-NODE  and  PATH 
(i.c..  (ACCESS  ROOT-NODE  PATH))  will  return  END-NODE. 

(SETF  (ACCESS*  ROOT-NODE  PATH)  NODE-LIST) 

Knsurcs  that  future  calls  to  ACCESS*  with  ROOT-NODE  and  PATH 
(i.c.,  (ACCESS+  ROOT-NODE  PATH))  will  return  NODE-LIST. 

(SETF  (ACCESS*  ROOT-NODE  PATH)  END-NODE) 

Knsurcs  that  future  calls  to  ACCESS*  with  ROOT-NODE  and  PATH 
(i.c.,  (ACCESS*  ROOT-NODE  PATH))  will  return  END-NODE. 

The  ACCESS  functions  differ  in  how  they  handle  steps  that  cannot  be  traced,  and  what  they  do  when  a 
path  description  fans  out.  If  ACCESS  or  ACCESS*  encounter  a  missing  link,  an  error  is  signalled.  ACCESS* 
and  the  SETF  functions  will  create  the  link  and  continue  tracing  the  path. 

A  fanout  condition  occurs  when  an  attempt  it  made  to  trace  from  a  :  MULTIPLE  arity  port  of  a  p-node,  or 
when  more  than  one  p-node  satisfies  the  role-name/process-typc-namc  constraint  tracing  from  a  g-node. 
ACCESS,  ACCESS*  and  their  associated  SETF  functions  signal  errors  if  fanout  is  encountered.  ACCESS* 
will  continue  tracing  down  all  paths  and  returns  the  list  of  nodes  that  satisfied  the  path  description.  When 
SETFed,  ACCESS-*-  will  signal  an  error  if  fanout  is  encountered  before  the  last  step  in  the  path  description. 

In  Lisp  Machine  Lisp  [Weinreb  and  Moon  81]  and  Common  Lisp  [Steele  84],  the  special  form  PUSH  can 
be  used  for  functions  that  arc  SETFable.  PUSH  can  be  used  to  add  a  g-node  to  a  port.  For  example: 

(PUSH  SOME-G-NODE  ( ACCESS-*-  P-NODE  SOME-PATH)) 
is  equivalent  to: 

(SETF  (ACCESS-*-  P-NODE  SOME-PATH) 

(CONS  SOME-G-NODE  (ACCESS*  P-NODE  SOME-PATH))) 

'The  SETF  forms  and  ACCESS*  can  make  additive  changes  to  the  graph.  When  a  function  needs  to  create 
a  g-node  and  link  it  to  a  p-node  port,  a  name  needs  to  be  synthesized  for  die  new  g-node.  The  name  of  each 
g-node  resembles  a  filename  in  diat  it  has  two  parts,  a  primary  name  and  an  extension.  In  order  to  synthesize 
a  g-node  name,  the  function  copies  the  primary  part  from  the  grain  attached  to  the  port  specified  as  the 
NAME-SOURCE  port  for  the  port  being  linked  to  (sec  the  paragraph  about  role  descriptions  in  chapter  4).  An 
error  is  signalled  if  a  function  needs  to  derive  a  g-node  name  to  link  to  a  port  that  has  no  NAME-SOURCE  port 
associated  with  it.  The  extension  of  a  g-node  name  is  derived  from  its  grain  type  object.  If  the  grain  type 
represents  files,  then  die  extension  is  die  dcfault-filcname-extcnsion,  otherwise,  it  is  the  name  of  the  grain 
type  itself. 


5.2  Request  Handlers 

Request  handlers  specify  the  task  graph  derivation  steps  that  can  he  taken  whenever  the  request  associated 
with  the  handler  has  been  made,  without  considering  any  reference  declarations.  Requests  are  identified  with 
request  signatures  (much  like  reference  signatures),  lurch  request  signature  contains  two  fields,  a  request 
name  and  a  grain  type  name.  I  dr  example  die  signature: 

<:C0MPILE  :LISP-SOURCE> 

identifies  the  handler  designed  to  build  part  of  die  task  graph  needed  to  accomplish  the  compilation  of  a  I  .isp 
source  grain.  I  hc  signature: 

< : YACC  :YACC-GRAMMAR> 

identifies  the  handler  that  will  build  part  of  the  task  graph  needed  to  invoke  yacc  on  a  grammar. 

Not  all  possible  signatures  will  have  handlers  defined  for  them.  For  example  the  request  signature: 

< : COMPILE  :LISP-BINARY> 

identifies  a  nonsensical  request. 

Pre-reference  request  handlers  are  used  to  construct  the  parts  of  a  task  graph  which  will  be  needed 
regardless  of  the  ramifications  of  references.  For  example,  in  order  to  model  die  compilation  of  some 
:LISP-SOURCE  grain.  G.LISP.  the  following  links  can  be  made  without  considering  any  references;  the 
g-node  representing  G .  LISP  should  be  linked  to  the  SOURCE  port  of  a  :  LISP-COMPILE  p-nodc,  and  then 
the  BINARY  port  of  diis  p-nodc  should  be  linked  to  a  g-nodc  representing  the  binary  version  of  G.  LISP  (i.c., 
G .  BIN). 


Post- reference  request  handlers  are  used  for  modeling  processing  that  can  only  be  deduced  after  the 
implications  of  the  references  are  added  to  the  task  graph.  At  this  time  it  has  not  been  necessary  to  use  a  post 
reference  handler,  however,  they  arc  included  because  there  may  be  situations  where  their  use  is  appropriate. 


Defining  Request  Handlers 

Request  handlers  arc  defined  with  DEFINE -REQUEST -HANDLER.  Calls  have  the  form: 

(DEFINE-REQUEST-HANDLER  (REQUEST  GRAIN-TYPE-NAME  PRE-OR-POST) 

(ARGS) 

&BODY  BODY) 

REQUEST  The  name  of  die  request  being  handled. 

GRAIN-TYPE-NAME 

The  type  of  the  grain  that  the  handler  is  for. 

PRE-OR-POST  :PRE  indicates  that  this  is  a  prc-rcfcrcncc  handler.  :POST  indicates  that  this  is  a  post 
reference  handler. 
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ARGS  The  names  of  the  variables  passed  to  die  handler.  There  must  be  at  least  one  element  in 

this  list.  The  first  ARG  will  be  bound  to  die  g-node  associated  with  the  request  when  BODY 
is  evaluated. 

BODY  The  forms  that  constitute  the  handler.  They  arc  evaluated  with  the  arguments  passed  to 

die  handler  bound  to  the  variables  named  in  ARGS. 

All  requests  made  by  users  have  a  single  argument,  the  name  of  the  module  that  the  request  is  intended  for. 
Handlers  may  also  make  requests,  and  dicse  requests  can  contain  mom  than  one  argument.  The  handlers  for 
the  :LOAD+  and  :INCLUDE+  tasks  presented  in  Appendix  I  arc  examples  of  handlers  using  additional 
arguments. 

f  igure  5-1  contains  die  request  handler  definitions  for  l.isp  compilation  and  loading.  The  first  handler  is 
invoked  when  a  :  COMPILE  request  is  made  on  a  :L  ISP-SOURCE  module.  It  uses  ACCESS*  to  ensure  that 
die  task  graph  being  derived  models  the  fact  that  die  : LISP-SOURCE  grains  in  the  module  need  to  be 
compiled. 

The  second  handler  is  invoked  when  a  rLOAD  request  is  made  on  a  :  LISP-SOURCE  module.  Ihc  first 
thing  that  the  handler  docs  is  to  initiate  a  :  COMPILE  request  on  each  of  the  grains  in  the  :  LISP-SOURCE 
module,  and  then  it  models  die  fact  that  the  .-BINARY  grains  produced  by  compilation  need  to  be  loaded. 

Handlers  ensure  that  task  graph  paths  exist.  After  a  handler  has  been  invoked  on  a  grain  once,  additional 
invocations  will  have  no  effect.  Therefore,  task  definers  need  only  be  concerned  that  the  proper  handlers  arc 
invoked  at  least  once  and  do  not  need  to  worry  about  additional  invocations. 

(DEFINE-REQUEST-HANDLER  (:COMPILE  :LISP-SOURCE  :PRE)  (SOURCE-NODE) 

(ACCESS*  SOURCE-NODE  ((SOURCE  : LISP-COMPILE )  BINARY))) 

(DEFINE-REQUEST-HANDLER  (-.LOAD  : LISP-SOURCE  :PRE)  (SOURCE-NODE) 
(PROCESS-REQUEST  :C0MPILE  SOURCE-NODE) 

(ACCESS*  SOURCE-NODE  ((SOURCE  : LISP-COMPILE )  BINARY 

(BINARY  : LISP-LOAD-BIN )  IMAGE))) 

Figure  5-1:  Request  Handler  Definitions  for  Lisp 

5.3  Reference  Handlers 

Reference  handlers  realize  the  implications  references  upon  construction  graphs.  The  construction 
implications  of  a  reference  depend  upon  the  kind  of  reference,  the  request,  and  which  part  of  the  reference 
(right  or  left)  the  module  participating  in  the  request  belongs  to.  Each  handler  is  identified  by  a  reference 
handler  signature  that  includes  five  fields:  the  three  fields  from  the  reference  signature,  the  request  name,  and 
a  participation  marker  (cither  :  RIGHT  or  :LEFT).  Sample  signatures  are: 

<< : CALLS  : LISP-SOURCE  : LISP-SOURCE>  < : LOAD  : LEFT>> 

<< : CALLS  : C-SOURCE  :C-SOURCE>  <:COMPILE  :RIGHT>> 

<< : MACRO-CALLS  :LISP-SOURCE  : LISP-SOURCE>  <:C0MPILE  : LEFT» 

Not  all  references  arc  relevant  to  every  request  made.  For  instance,  the  reference 

( : CALLS  LISP-SOURCE-1  LISP-SOURCE -2 ) 

has  no  implications  when  a  request  is  made  to  compile  LISP-SOURCE-1.  However,  if  the  request  is  to  load 
LISP-SOURCE-1  for  execution,  then  the  reference  implies  that  LISP-SOURCE-2  needs  to  be  loaded.  It  is 
also  important  to  recognize  dial  the  direction  of  the  reference  matters.  For  example,  the  reference  above  has 
implications  when  LISP-SOURCE-1  is  loaded,  but  it  has  none  when  LISP-SOURCE-2  is  loaded. 


Defining  Reference  Handlers 

Reference  handlers  arc  defined  with  DEFINE-REFERENCE-HANDLER.  Calls  have  the  form: 

( DEF I NE -REFERENCE -HANDLER  ((REFERENCE  LEFT-TYPE  RIGHT-TYPE) 

(REQUEST  DIRECTION)) 

(ARGS) 

&B0DY  BODY) 

REFERENCE  Hie  name  of  the  reference  being  handled. 

LEFT-TYPE  The  grain  type  of  the  left  (first)  module  in  the  reference. 

RIGHT- TYPE  The  grain  type  of  the  right  (second)  module  in  the  reference. 

REQUEST  The  name  of  the  request  being  handled. 

DIRECTION  father  :LEFT  or  :  RIGHT.  This  field  identifies  die  module  that  the  request  being  handled 
refers  to. 

ARGS  The  names  of  die  variables  passed  to  die  handler,  these  will  be  hound  when  BODY  is 

evaluated.  There  must  be  at  least  two  elements  in  diis  list.  The  first  ARG  will  be  bound  to 
die  left  grain  of  the  reference.  The  second  ARG  will  be  bound  to  the  right  grain  of  the 
reference. 

BODY  The  forms  that  constitute  the  handler.  They  arc  evaluated  with  the  arguments  passed  to 

the  handler  bound  to  the  variables  named  in  ARGS. 

Figure  5-2  contains  reference  handler  definitions  for  Lisp  compilation  and  loading.  The  first  handler 
models  the  fact  that  the  grain  represented  by  CALLED-NODE  needs  to  be  loaded,  and  that  the  resulting 
:  LISP-IMAGE  grain  plays  the  role  DEFINITIONS  in  the  compilation  of  the  grain  represented  by 
CALLING-NODE.  The  second  handler  ensures  that  the  grain  represented  by  :CALLED-NODE  is  loaded. 
Note,  while  these  handlers  arc  sufficient  to  handle  the  common  module  interactions  for  Lisp  systems,  they  are 
not  sufficient  to  handle  all  of  the  ways  that  Lisp  modules  may  interact.  More  handlers  would  need  to  be 
defined  in  order  to  properly  handle  all  of  the  ways  that  Lisp  modules  can  interact.  The  prototype 
implementation  of  BUILD  does  not  include  these  additional  handlers  at  this  time. 

BL ll  n  guarantees  that  reference  handlers  arc  invoked  after  prc-rcfcrcncc  request  processing  and  therefore 
handler  w  riters  may  safely  assume  that  the  effects  of  pre-reference  request  handlers  will  already  be  present  in 
the  graph.  For  example,  the  :MACRO-CALLS  handler  discussed  above  assumes  that  the  compilation  of 
CALLING-NODE  has  already  been  modeled. 

5.4  A  Task  Description  Definition  Example 

This  section  presents  an  example  of  a  task  description  definition.  The  task  defined  is  called 
:  L  1ST -SOURCE  -CODE  and  it  will  produce  formatted  source  code  listings  for  a  :  LISP-SOURCE  module  and 
any  :LISP-SOURCE  modules  that  it  references.  All  of  die  defining  forms  for  :  LIST-SOURCE-CODE  arc  in 
figure  5-3. 

First,  the  :  L  1ST -LISP-SOURCE  process  type  is  defined.  Instances  of  this  type  have  a  single  input  role 
called  SOURCE  and  a  single  output  role  called  LISTING.  The  function  LIST-LISP-FILE  is  called  to 
produce  the  grain  filling  the  output  role  from  llic  grain  filling  the  input  role.  T  he  request  handler  for  die  task 
is  very  simple,  it  models  the  fact  diat  die  source  grain  to  be  lisled  will  play  die  role  SOURCE  in  a 
:  L IST-L ISP-SOURCE  p-node  and  that  a  g-nodc  should  be  attached  to  the  LISTING  role  of  that  same 


p-node.  The  two  reference  handlers  specify  that  grains  which  arc  called  by  a  grain  being  listed  should 
themselves  he  listed. 

:LIST-SOURCE-CODE  shows  the  virtue  of  keeping  system  models  separate  from  information  about 
tasks:  once  its  defining  forms  arc  evaluated,  formatted  listings  may  be  obtained  for  any  previously  modeled 
I  isp  system  without  altering  any  system  models. 


(DEFINE-REFERENCE-HANDLER  ( ( :MACRO-CALLS  :LISP-SOURCE  : LISP-SOURCE) 

{ : COMPILE  :LEFT ) ) 

(CALLING-NODE  CALLED-NODE) 

(PROCESS-REQUEST  :L0A0  CALLEO-NODE) 

(PUSH  (ACCESS  CALLED-NODE  ((SOURCE  : L ISP-COMPILE )  BINARY 

(BINARY  :LISP-LOAD-BIN)  IMAGE)) 

( ACCESS+  CALLING-NODE  ((SOURCE  : LISP-COMPILE )  DEFINITIONS)))) 

(DEFINE-REFERENCE-HANDLER  ( ( : CALLS  :LISP-SOURCE  : LISP-SOURCE) 

( : LOAD  : LEFT ) ) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  : LOAD  CALLED-NODE)) 

Figure  5-2:  Reference  Handler  Definitions  for  1. isp 


(DEFINE-PROCESS-TYPE  : LIST-LISP-SOURCE 
((SOURCE  : LISP-SOURCE  :SINGLE)) 

((LISTING  : PRESS  :SINGL£  SOURCE)) 

OUTPUT-STREAM 

(FORMAT  OUTPUT-STREAM  "-XLIST  -A" 

(PATHNAME-MINUS-VERSION  SOURCE)) 

(FORMAT  OUTPUT-STREAM  "~%LISTING  ~A"  SOURCE) 

(LIST-LISP-FILE  SOURCE  LISTING)) 

(DEFINE-REQUEST-HANDLER  (: LIST-SOURCE-CODE  :LISP-SOURCE  :PRE) 

(SOURCE -NODE) 

(ACCESS*  SOURCE-NODE  ((SOURCE  : LIST-LISP-SOURCE )  LISTING))) 

(DEFINE-REFERENCE-HANDLER  ((.-MACRO-CALLS  :LISP-SOURCE  : LISP-SOURCE) 

(: LIST-SOURCE-CODE  : LEFT ) ) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  : LIST-SOURCE-CODE  CALLED-NODE)) 

(DEFINE-REFERENCE-HANDLER  ((.CALLS  :LISP-SOURCE  :LISP-SOURCE) 

( -.LIST-SOURCE-CODE  -.LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  : LIST-SOURCE-CODE  CALLED-NODE)) 

Figure  5-3:  Definition  For  : LIST-SOURCE-CODE 
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6.  Reprise 


'ITiis  chapter  highlights  several  aspects  of  HUM  I)  that  have  been  presented  in  this  report.  1'hc  first  section 
summarizes  how  iu.il  D  overcomes  the  difficulties  associated  with  existing  tools  (sec  chapter  2).  1'hc  second 
section  discusses  null  l)‘s  construction  framework  and  how  it  provides  a  base  for  describing  new  tasks  within  a 
static  framework  that  conceals  many  low  level  details  from  the  task  defincr.  The  final  section  proposes  ways 
that  null  n  could  be  extended  to  provide  capabilities  not  found  in  existing  tools. 

6.1  BUILD  Compared  With  Existing  Tools 

Phrased  in  terms  of  inter-module  references.  The  null  D  system  modeling  mechanism  allows  users  to 
describe  systems  in  terms  that  arc  natural  for  them,  null  D  system  models  arc  easier  to  understand  and  they 
provide  more  information  than  the  construction  directive  lists  used  by  existing  tools. 

User  definable  task  descriptions.  build’s  task  description  mechanism  is  responsible  for  the  fact  that  BUILD 
is  not  constrained  to  some  embedded  set  of  tasks.  Ity  separating  system  models  and  task  descriptions,  build’s 
knowledge  about  construction  can  be  modified  without  requiring  that  system  models  be  changed.  However, 
if  a  new  task  is  sensitive  to  a  class  of  references  previously  ignored,  then  existing  models  will  have  to  be 
updated. 

Intermediate  grains  arc  not  referenced.  The  only  grains  that  arc  referred  to  in  a  system  arc  die  source 
grains  that  comprise  modules.  While  intermediate  grains  arc  used  in  nuil.D's  task  graphs,  these  grains  never 
appear  in  system  models. 

All  source  grains  must  be  referenced.  All  of  the  source  grains  that  participate  in  a  system  cither  reference 
other  grains  in  the  system  or  are  referenced  by  other  grains  in  the  system.  Therefore,  since  BUILD  models 
encode  system  referencing  patterns,  all  of  the  source  grains  in  a  system  must  appear  in  any  well  formed  BUIIJD 
model  of  that  system. 

6.2  BUILD’s  Construction  Framework 

build  provides  procedures  which  guide  the  construction  process.  These  procedures  include  hooks  for  the 
components  of  user  supplied  task  descriptions.  The  set  of  fixed  procedures  take  care  of  low  level  construction 
details  that  arc  common  to  all  tasks  and  allow  task  definitions  to  contain  just  the  details  that  arc  relevant  to  the 
particular  task  being  defined. 

The  task  graph  representation  and  analysis  algorithm  provide  a  uniform  way  to  describe  and  perform 
system  maintenance  tasks.  New  process  types  and  grain  types  can  easily  be  integrated  into  task  graphs. 

The  ACCESS  family  of  functions  provide  a  general  way  for  viewing  and  manipulating  task  graphs  that 
isolates  handler  definitions  from  the  low  level  mechanics  of  instantiating  nodes,  matching  grain  types  between 
g-nodcs  and  p-nodc  ports,  and  actually  linking  nodes  together. 

The  task  graph  derivation  algorithm  ensures  that  pre-reference  request  handlers  are  invoked  before 
reference  handlers  and  that  reference  handlers  are  invoked  before  post-rcfcrcncc  request  handlers.  This 
algorithm  is  also  responsible  for  translating  module  references  into  a  scries  of  handler  invocations,  one  for 
each  grain  involved  in  a  reference.  Finally,  the  task  graph  derivation  algorithm  ensures  that  circular 
references  (i.c.,  (:  CALLS  A  B)  (:CALLS  B  A))  do  not  cause  infinite  loops  during  reference  handling. 

null  D’s  construction  framework  allows  task  definers  to  concentrate  on  the  significant  details  of  the  task 
being  defined  (c.g..  what  process  and  grain  types  arc  used,  what  references  are  relevant  and  how  should  they 
be  handled  etc.)  and  isolates  them  from  low  level  details  (c.g.,  task  graph  analysis,  node  instantiation  etc.). 


6.3  Extensions  to  BUILD 

ir  11 1)  provides  a  more  graceful  way  of  modeling  systems  than  existing  tools,  yet  it  does  not  provide 
greater  capabilities.  This  section  proposes  extensions  to  m  il  n  that  would  allow  it  to  provide  a  set  of  facilities 
that  other  tools  do  not.  The  extensions  arc  automatic  derivation  of  system  specifications  from  source  code, 
support  for  patching  and  similar  maintenance  styles,  and  the  incorporation  of  the  nature  of  module  change 
into  the  reconstruction  algorithms. 

Automatic  Derivation  of  System  Descriptions 

The  m  ll  i)  modeling  mechanism  provides  a  natural  way  to  describe  systems  but  it  docs  not  ensure  that  the 
descriptions  arc  complete  or  correct.  Designers  arc  still  required  to  generate  system  models  by  hand.  A  tool 
that  could  derive  system  models  from  source  code  would  relieve  designers  of  the  chore  of  building  system 
description  files. 

For  simple  languages,  an  analyzer  could  build  a  great  deal  of  the  model  and  locate  areas  that  might 
present  difficulties.  For  example,  in  most  C  systems  all  of  the  dependencies  arc  caused  by  use  of  the 
^INCLUDE  compiler  directive  and  calls  to  externally  defined  symbols  --  the  reference  assertions  from  these 
references  could  be  synthesized  automatically. 

While  there  may  be  programming  environments  in  which  it  is  possible  to  mechanize  the  derivation  of 
system  models  there  arc  certainly  languages  for  which  such  derivation  would  become  arbitrarily  complex. 
For  example,  Piunan  develops  an  argument  against  automatic  derivation  of  Lisp  system  models  based  on  the 
complications  caused  by  macros  [Pitman  84J. 

Patching 

There  arc  many  instances  where  a  system  maintaincr  may  want  to  introduce  changes  into  a  system  without 
making  sure  that  the  resulting  system  is  consistent:  for  example,  debugging  experiments  where  small  changes 
arc  introduced  to  examine  some  small  part  of  th<*  system,  These  changes  may  not  be  intended  to  become  part 
of  a  released  system,  it  may  even  be  known  that  they  will  cause  compilation  of  some  other  module  to  fail. 
Another  instance  where  the  ability  to  patch  a  system  is  important  is  when  a  quick  fix  is  being  attempted  and  it 
is  important  that  the  effects  be  seen  quickly.  This  kind  of  change  represents  a  tentative  guess  on  the  part  of 
the  maintaincr.  The  introduction  of  such  changes  into  systems  must  be  supported  by  system  management 
tools  if  such  tools  arc  going  to  help  and  not  hinder  maintained. 

The  DHI'SYSTP.M  patch  facility  provides  some  support  for  producing  inconsistent  systems.  Unfortunately, 
the  defsystpm  patching  facility  makes  no  use  of  the  dependency  information  that  the  rest  of  the  tool  uses. 
No  analysis  of  the  effect  of  a  patch  is  available.  Nothing  guarantees  that  a  patch  will  ever,  be  loaded  correctly 
according  to  the  dependency  information  that  is  available.  For  example,  if  a  patch  file  includes  a  modified 
macro  definition  and  two  calls  to  it,  the  calls  will  not  refer  to  the  new  version  of  the  macro  unless  they  are 
placed  after  the  definition  in  the  patch  file  by  the  user. 

System  management  tools  should  make  use  of  system  models  in  order  to  support  patching.  Patching 
mechanisms  should  also  supply  information  about  the  effect  that  a  patch  may  have  on  the  rest  of  the  system. 
In  null  D.  the  analysis  could  be  done  by  propagating  the  effects  of  a  change  through  a  task  graph  and  then 
identifying  those  modules  that  were  affected  by  the  change  but  ignored  by  the  patch. 

More  Precise  Change  Analysis 

All  of  the  tools  mentioned  in  this  paper  (including  build)  arc  sensitive  to  the  fact  that  some  change  has 
occurred  to  a  module  in  a  system.  However,  no  attention  is  paid  to  the  nature  of  the  change.  By  exploring  the 
nature  of  a  change  it  is  possible  to  limit  the  amount  of  processing  done  when  updating  systems. 

If  source  code  is  changed  in  a  way  that  cannot  alter  its  compilation,  there  is  no  reason  for  die  source 
module  to  be  recompiled.  For  example,  compilation  should  not  be  done  when  source  code  has  only  been 


reformatted  or  had  commentary  added  to  it.  If  a  function  is  added  to  a  module,  hut  no  existing  modules  arc 
updated  to  contain  calls  to  the  new  function,  nothing  should  be  done  to  the  existing  modules,  l.int  libraries 
are  dependent  upon  the  first  pass  of  l  int,  however,  most  changes  to  the  first  pass  of  l.int  will  not  affect  the 
libraries. 

Change  analysis  can  also  provide  important  debugging  information.  For  example,  if  a  module  interface  is 
changed,  but  not  all  of  the  modules  that  contain  references  to  that  module  are  changed,  there  is  a  possibility 
that  an  error  of  omission  has  been  made. 

Unlike  MAKI-:  and  HUH  l),  DllSYSllM  can  be  extended  to  include  more  complicated  predicates  for 
deciding  when  changes  arc  significant.  There  is  nothing  preventing  a  DI'lSYSll  M  system  definition  from 
using  parsers  and  source  code  comparison  programs  in  order  to  decide  when  transformations  should  take 
place.  However,  no  enhanced  predicates  arc  supplied  with  ditsysti-m  and  none  of  the  DWSYSTHM 
descriptions  encountered  while  preparing  this  paper  included  definitions  of  such  specialized  predicates. 

Specialized  predicates  can  only  be  useful  if  they  require  less  processing  to  determine  that  a  transformation 
can  be  avoided  than  applying  the  transformation  in  the  first  place.  For  instance,  there  is  no  point  in  using  a 
predicate  to  determine  that  compilation  of  a  module  can  be  avoided  if  that  predicate  requires  more  processing 
titan  the  compiler,  huh  i>  can  step  around  this  issue  by  assuming  that  it  is  a  single  tool  embedded  in  an 
integrated  environment  in  which  die  tools  that  arc  used  to  modify  modules  can  supply  information  to  BUII.D 
about  the  nature  of  changes. 

HUH  l)  could  be  extended  to  provide  an  interface  for  communicating  information  about  changes  to 
modules.  The  information  passed  to  BUII.D  would  include  the  name  of  the  grain  modified,  the  kind  of 
modification  made,  and  the  name  of  the  new  (i.e..  updated)  grain.  A  new  class  of  handlers  called  change 
handlers  would  be  introduced  to  aid  in  the  determination  of  significant  changes  by  the  construction  algorithm. 

For  example,  the  change  assertion: 

( : ADDED-STRUCT  DEFS) 

would  inform  BUII  n  that  DEFS  has  been  changed  by  adding  a  new  structure  and  therefore  modules  that  rely 
on  DE  FS  do  not  have  to  be  re-compiled.  The  compilation  of  unaltered  modules  can  be  avoided  since  there  is 
no  way  for  them  to  refer  to  the  new  structure.  The  assertions: 

( : ADDED-COMMENT  DEFS) 

(: RE-FORMATTED  DEFS) 

imply  that  no  changes  that  can  alter  the  compilation  of  DEFS  have  been  made  and  therefore  no  re¬ 
compilation  needs  to  be  done. 

The  change  handlers  would  contain  listings  of  how  types  of  changes  alter  the  way  in  which  grains  play 
their  roles.  For  instance,  one  handler  would  note  that  re-formatting  a  piece  of  source  code  does  not  change 
the  way  that  it  plays  the  role  SOURCE  in  instances  of  :  LISP-COMPILE. 
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I.  BUILD  Definitions  For  C 


The  definitions  used  by  bum  i>  to  model  a  l.isp  environment  have  been  given  in  the  body  of  this  report  as 
examples.  This  appendix  contains  die  definitions  used  by  null  I)  to  model  a  C  programming  environment. 
There  are  more  kinds  of  commonly  used  grain  types  in  UNIX  environments  than  in  l.isp  environments, 
hence  there  arc  more  definitions  needed  to  model  all  of  die  ways  that  UNIX  grains  can  refer  to  each  other. 
Commentary  has  been  added  to  highlight  the  definitions. 


Grain  Type  Definitions 


(Dt FINE-GRAIN-TYPE 
(DEFINE-GRAIN-TYPE 
(DEF1NE-GRAIN-TYPE 
(DEFINE-GRAIN-TYPE 
(DEFINE-GRAIN-TYPE 


:YACC -GRAMMAR  :Y) 

: C-SOURCE  :C) 

: C-OBJECT  :0) 

: C-EXECUTE  : EXE ) 

: SHELL-SCRIPT  :SCRIPT) 


Process  Type  Definitions 

It  is  assumed  dial  the  functions  C-COHPILE ,  C-LOAD.  and  YACC  arc  available. 

(DEFINE-PROCESS-TYPE  C-COMPILE 

((SOURCE  : C-SOUKCE  : SINGLE )  (INCLUDES  :C-S0URCE  : MULTIPLE ) ) 
((OBJECT  : C-OBJECT  : SINGLE  SOURCE)) 

STREAM 

(FORMAT  STREAM  "-XCOMPILE  -A"  (PATHNAME-MINUS-VERSION  SOURCE)) 
(FORMAT  STREAM  "-XCOMPILING  -A"  SOURCE) 

(C-COMPILE  SOURCE  OBJECT)) 

(DEFINE-PROCESS-TYPE  C-LOAD 

((PRIMARY  : C-OBJECT  :SINGLE )  (SECONDARY  :C-OBJECT  :MULTIPLE)) 
((IMAGE  : C-EXECUTE  : SINGLE  PRIMARY)) 

STREAM 

(FORMAT  STREAM  "-XLINK:  -A  -{-X  -A-}" 

(PATHNAME-MINUS-VERSION  PRIMARY) 

(MAPCAR  PATHNAME -MINUS -VERSION  SECONDARY)) 

(FORMAT  STREAM  "-^LINKING:  -A  ~{-X  -A-}"  PRIMARY  SECONDARY) 

(C-LOAD  PRIMARY  SECONDARY  IMAGE)) 

(DEFINE-PROCESS-TYPE  YACC 

((GRAMMAR  : YACC-GRAMMAR  : SINGLE ) ) 

((PARSER  : C-SOURCE  :SINGLE  GRAMMAR)) 

STREAM 

(FORMAT  STREAM  "-XYACC  -A"  (PATHNAME-MINUS-VERSION  GRAMMAR)) 
(FORMAT  STREAM  "-XYACCING  -A"  GRAMMAR) 

(YACC  GRAMMAR  PARSER)) 


Request  and  Reference  Handlers 

The  request  handler  for  C  compilation  models  the  fact  that  the  source  grain  needs  to  be  compiled.  'ITtc 
only  reference  that  can  have  an  effect  on  C  compilation  is  :  INCLUDES.  If  GRAIN- 1  includes  GRAIN-2, 
then  GRAIN- 1  indirectly  includes  any  grains  that  GRAIN-2  includes.  The  task  :  I NCLUDE+ (described  later) 
is  responsible  for  gathering  all  of  the  grains  included  indirectly  by  a  grain  and  attaching  the  corresponding 
g-nodcstolhc  INCLUDES  port  of  the  -.C-COMPILE  p-nodc  for  the  grain  being  compiled. 


: COMPILE  :C-S0URC£ 


(DEFINE -REQUE ST -HANDLER  (:COMPILE  :C-S0URCE  :  PRE )  (SOURCE-NODE) 

(ACCESS*  SOURCE-NODE  ((SOURCE  C-COMPILE)  OBJECT))) 

(DEFINEREFERENCE-HANDLER  ( ( : INCLUDES  :C-SOURC£  :C-SOURCE)  (:C0MP1LE  .LEFT)) 

( INCLUDING-NODE  INCI UDED-NODE ) 

(LET  ((C0MPI1 E-PROCESS  (ACCtSS  INCLUDING-NODE  ((SOURCE  C-COMPILE))))) 

(PUSH  INCLUDED -NODE  (ACCISS+  COMPILE -PROCESS  (INCLUDES))) 
(PROCESS-REQUEST  :INCLUDL+  INCLUDED-NODE  COMPILE -PROCESS) ) ) 


If  a  :C-SOURCE  grain  calls  another  grain,  then  HUH  l)  pessimistically  assumes  that  it  indirectly  calls  any 
grain  called  by  the  second  grain.  The  task  :  LOAD+  gathers  all  of  the  grains  called  indirectly  by  a  grain  in 
order  to  ensure  that  the  proper  set  of  grains  is  linked  together.  The  lack  of  a  task  like  :  LOAD*  in  Lisp  is  due 
to  the  fact  that  in  I  isp  environments,  grains  arc  loaded  incrementally  instead  of  being  explicitly  linked 
together. 


:  LOAO  : C-SOURCE 


(DEFINE -REQUE ST -HANDLER  ( : LOAD  :C*SOURCE  :PRE)  (SOURCE-NODE) 

(PROCESS-REQUEST  rCOMPILE  SOURCE-NODE) 

(ACCESS*  SOURCE-NODE  ((SOURCE  C-COMPILE)  OBJECT  (PRIMARY  C-LOAD)  IMAGE))) 

(OE FINE -REFERENCE -HANDLER  ((:CALLS  :C-SOURCE  :C-SOURCE)  ( : LOAD  :LEFT)) 

(CALLING-NODE  CALLED-NODE) 

(LET  ((LINKING-PROCESS 

(ACCESS  CALLING-NODE  ((SOURCE  C-COMPILE)  OBJECT  (PRIMARY  C-LOAD))))) 
(PROCESS-REQUEST  : COMPILE  CALIEO-NODE) 

(PUSH  (ACCESS  CALLED-NODE  ((SOURCE  C-COMPILE)  OBJECT)) 

(ACCESS*  LINKING-PROCESS  (SECONDARY))) 

(PROCESS-REQUEST  :L0AD*  CALLED-NODE  LINKING-PROCESS))) 

(DEFINE-REFERENCE-HANDLER  ((:CALLS  :C-S0URCE  :C-OBJECT)  ( : LOAD  :LEFT)) 

(CALLING-NODE  CALLED-NODE) 

(LET  ((LINKING-PROCESS 

(ACCESS  CALLING-NOOE  ((SOURCE  C-COMPILE)  OBJECT  (PRIMARY  C-LOAD))))) 

(PUSH  CALLED-NODE  (ACCESS*  LINKING-PROCESS  (SECONDARY))) 

(PROCESS-REQUEST  : LOAD*  CALLED-NOOE  L INKING- PROCESS )) ) 

Sometimes  compiled  objects  arc  used  as  source  grains  (c.g.  supplied  libraries).  These  definitions  encode 
the  knowledge  needed  to  handle  the  loading  of  :  C-OB  JECT  grains. 


: LOAO  :C-OBJECT 


(DEFINE-REQUEST-HANDLER  ( : LOAD  C-OBJECT  :PRE)  (OBJECT-NOOE ) 
(ACCESS*  OBJECT-NOOE  ((PRIMARY  C-LOAO)  IMAGL ) ) ) 
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(DEflNt-RtffRENCE  -  HANDIER  ((:CALIS  :C-OBJECT  :C-SOURCE)  (  :  LOAD  :LEET  )) 

( CAL  I  I NG-NODt  CALLED-NODE) 

(IET  ((II NK1NG- PROCESS  (ACCESS  CAIL 1NG-NODE  ((PRIMARY  C-LOAD))))) 

( PROCLSS-REQUE  SI  :COMP!Lf  CALLED-NODE) 

(PUSH  (ACCESS  CAtLED-NODE  ((SOURCE  C-COMP1LE)  OBJECT)) 

(ACCESS-*  LINKING-PROCESS  (SECONDARY))) 

( PROCESS -RE OUE  SI  :LOAD+  CALLED-NODE  LINKING-PROCESS))) 

(DEE  I  ME  -RLf  ERE  NCE -HANDLER  ((:CALLS  -.C-OBJECT  :C-OBJECT)  (  :  LOAD  :  LEET  ) ) 

(CALLING-NODE  CALLED-NODE) 

(LET  ((LINKING-PROCESS  (ACCESS  CALLING-NODE  ((PRIMARY  C-LOAD))))) 
(PUSH  CALLED-NODl  (ACCESS+  L INKING- PROCESS  (SECONDARY))) 
(PROCESS-REQUEST  -. LOAD+  CALLED-NODE  LINKING-PROCESS))) 


Here  are  the  handlers  for  :  INCLUDE+  and  :  LOAD+.  There  arc  no  request  handlers  associated  with  these 
requests  as  all  of  the  significant  construction  information  that  they  imply  arises  from  references.  These 
handlers  illustrate  the  use  of  more  than  two  values  being  passed  to  reference  handlers.  Ihc  additional 
parameter  for  :INCLUDE+  is  the  :C-COMPILE  p-nodc  which  models  die  compilation  to  be  done.  The 
additional  parameter  for  :  LOAD+  is  the  p-node  which  models  the  linking  to  be  done. 


: INCLUDE*  :C-SOURCE  C-COMPILE 


(DEFINE-REFERENCE-HANDLER  ( (  :  INCLUDES  :C-SOURCE  :C-SOURCE)  (  :  INCLUDE-*-  :LEFT)) 

(IGNORE  INCLUDED-NODE  INCLUDING-PROCESS) 

(PUSH  INCLUDED-NODE  ( ACCESS+  INCLUDING-PROCESS  (INCLUDES))) 

(PROCESS-REQUEST  : INCLUDE*  INCLUDED-NODE  INCLUDING-PROCESS)) 


: LOAD-*-  .C-SOURCE  C-LOAD 


(DEFINE-REFERENCE-HANDLER  ((:CALLS  :C-SOURCE  :C-SOURCE)  <:LOAD+  :  LEFT ) ) 

(IGNORE  CALLED-NODE  LINKING-PROCESS) 
(PROCESS-REQUEST  :COMPILE  CALLED-NODE) 

(PUSH  (ACCESS  CALLED-NODE  ((SOURCE  C-COMPILE)  OBJECT)) 

(ACCESS+  LINKING-PROCESS  (SECONDARY))) 

(PROCESS-REQUEST  . LOAD-*-  CALLED-NODE  LINKING-PROCESS)) 

(DEFINE-REFERENCE-HANDLER  ( ( rCALLS  :C-SOURCE  :C-OBJECT)  (:LOAD+  : LEFT) ) 

(IGNORE  CALLED-NODE  LINKING -PROCESS) 

(PUSH  CALLED-NODE  (ACCESS-*-  LINKING-PROCESS  (SECONDARY))) 
(PROCESS-REQUEST  : LOAD-*-  CALLEO-NODE  LINKING-PROCESS)) 


: L0AD+  : C-OBJECT  C-LOAD 


(DEFINE-REFERENCE-HANDLER  ((:CALLS  :C-OBJECT  :C-SOURCE)  (:LOAD+  : LEFT ) ) 

(IGNORE  CALLED-NODE  LINKING-PROCESS) 
(PROCESS-REQUEST  :COMPIlE  CALLED-NODE) 

(PUSH  (ACCESS  CALLED-NODE  ((SOURCE  C-COMPILE)  OBJECT)) 

(ACCESS-*  LINKING-PROCESS  (SECONDARY))) 

(PROCESS-REQUEST  :LOAD+  CALLED-NODE  LINKING-PROCESS)) 

(DEFINE-REFERENCE-HANDLER  ( ( : CALLS  :C-OBJECT  :C-OBJECT)  (:LOAD+  : LEFT ) ) 

(IGNORE  CALLED-NODE  LINKING-PROCESS) 

(PUSH  CALLED-NODE  (ACCESS-*  LINKING -PROCESS  (SECONDARY))) 
(PROCESS-REQUEST  : LOAD-*  CALLED-NODE  LINKING-PROCESS)) 
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Here  are  the  definitions  used  to  model  yacc's  interaction  with  C  systems.  I  he  handlers  capture  the  fact 
that  yac  v  grammars  may  include  and  call  other  grains. 


: YACC  : YACC-GRAMMAR 


(DfriNE-Rf QUEST-HANDLER  ( : YACC  : YACC-GRAMMAR  : PRE  )  (GRAMMAR-MODE) 
(ACCESS*  GRAMMAR-MODE  ((GRAMMAR  YACC)  PARSER))) 


: COMPILE  : YACC-GRAMMAR 


(DE El NE -REQUEST -HANDLER  (  COMPILE  : YACC-GRAMMAR  : PRE )  (GRAMMAR-MODE) 

( PROCl SS-REQUl ST  :YACC  GRAMMAR-NODE) 

(ACCESS*  GRAMMAR-NODE  ((GRAMMAR  YACC)  PARSER  (SOURCE  C-COMPILE)  OBJECT))) 

(DEFINE -REEERENCE -HANOI ER  ( ( : INCLUDES  : YACC-GRAMMAR  :C-SOURCE)  (:COMPILE  : LEFT ) ) 

(  INCLUDING -NODE  INCLUDED-NODE ) 

(IET  ((COMPIlE-PROCESS 

(ACCESS  INCLUDINC-NOOE  ((GRAMMAR  YACC)  PARSER  (SOURCE  C-COMPIl E ))))) 
(PUSH  INCLUDED-NODE  (ACCESS+  COMPILE -PROCESS  (INCLUDES))) 

(PROCL SS-REQUL SI  : INCLUDE*  INCLUDED-NODE  COMPILE-PROCESS))) 


:  LOAD  : YACC-GRAMMAR 


(DEFINE -REQUEST -HANDLER  ( : LOAD  : YACC-GRAMMAR  :PRE)  (GRAMMAR-NODE) 
(PROCESS-REQUEST  -.COMPILE  GRAMMAR-NODE) 

(ACCESS*  GRAMMAR-NODE  ((GRAMMAR  YACC)  PARSER 

(SOURCE  C-COMPILE)  OBJECT 
(PRIMARY  C-LOAD)  IMAGE))) 

(DEFINE-REFERENCE-HANDLER  ((:CALLS  : YACC-GRAMMAR  :C-SOURCE)  ( : LOAD  : LEFT) ) 

(CALLING-NODE  CALLED-NODE) 

(LET  ((LINKING-PROCESS  (ACCESS  CALLING-NODE  ((GRAMMAR  YACC)  PARSER 

(SOURCE  C-COMPILE)  OBJECT 
(PRIMARY  C-LOAD))))) 

(PROCESS-REQUEST  : COMPILE  CALLED-NODE) 

(PUSH  (ACCESS  CALLED-NODE  ((SOURCE  C-COMPILE)  OBJECT)) 

(ACCESS*  LINKING-PROCESS  (SECONDARY))) 

(PROCESS-REQUEST  :LOAD+  CALLED-NOOE  LIHKING-PROCESS))) 

(DEFINE-REFERENCE-HANDLER  ( ( : CALLS  : YACC-GRAMMAR  :C-0BJECT)  ( : LOAD  : LEFT) ) 

(CALLING-NODE  CALLED-NODE) 

(LET  ((LINKING-PROCESS  (ACCESS  CALLING-NODE  ((GRAMMAR  YACC)  PARSER 

(SOURCE  C-COMPILE)  OBJECT 
(PRIMARY  C-LOAD))))) 

(PUSH  CALLED-NODE  (ACCESS*  LINKING-PROCESS  (SECONDARY))) 
(PROCESS-REQUEST  : LOAD*  CALLED-NODE  LINKING-PROCESS))) 


Here  arc  the  definitions  used  to  handle  :  SHELL-SCRIPT  grains.  A  request  to  compile  or  load  a  shell 
script  is  interpreted  to  mean  that  all  of  the  modules  called  by  the  script  should  be  compiled  or  loaded. 


(COMPILE  :SHtLL-SCRIPT 


(DE FINE -RE f ERE NCE -HANDLER  (((CALLS  (SHELL-SCRIPT  (C-SOURCE)  ((COMPILE  (LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  .COMPILE  CALLED-NODE)) 

(DEFINE-REFERENCE-HANDLER  (((CALLS  (SHELL-SCRIPT  (C-OBJECT)  ((COMPILE  (LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  (COMPILE  CALLED-NODE)) 

(DEFINE-REFERENCE-HANDLER  (((CALLS  (SHELL-SCRIPT  : YACC-GRAMMAR )  ((COMPILE  (LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  (COMPILE  CALLED-NODE)) 


(LOAD  SHELL-SCRIPT 


(DEFINE-REFERENCE-HANDLER  ((CALLS  (SHELL-SCRIPT  (C-SOURCE)  ((LOAD  (LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  (LOAD  CALLED-NODE ) ) 

(DEFINE-REFERENCE-HANDLER  (((CALLS  (SHELL-SCRIPT  (C-OBJECT)  ((LOAD  (LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  : LOAO  CALLED-NODE)) 

(DEFINE-REFERENCE-HANDLER  (((CALLS  (SHELL-SCRIPT  : YACC-GRAMMAR )  ((LOAD  (LEFT)) 

(IGNORE  CALLED-NODE) 

(PROCESS-REQUEST  (LOAD  CALLED-NODE)) 


